------------------------------------------------------------------- Thu Jan 28 12:31:50 UTC 2021 - Ben Greiner - Update to version 2021.1.1 * Make system monitor interval configurable (GH#4447) Matthew Rocklin * Add uvloop config value (GH#4448) Matthew Rocklin * Additional optimizations to stealing (GH#4445) jakirkham * Give clusters names (GH#4426) Jacob Tomlinson * Use worker comm pool in Semaphore (GH#4195) Florian Jetter * Set runspec on all new tasks to avoid deadlocks (GH#4432) Florian Jetter * Support TaskState objects in story methods (GH#4434) Matthew Rocklin * Support missing event loop in Client.asynchronous (GH#4436) Matthew Rocklin * Don’t require network to inspect tests (GH#4433) Matthew Rocklin - Changelog for 2021.1.0 * Add time started to scheduler info (GH#4425) Jacob Tomlinson * Log adaptive error (GH#4422) Jacob Tomlinson * Xfail normalization tests (GH#4411) Jacob Tomlinson * Use dumps_msgpack and loads_msgpack when packing high level graphs (GH#4409) Mads R. B. Kristensen * Add nprocs auto option to dask-worker CLI (GH#4377) Jacob Tomlinson * Type annotation of _reevaluate_occupancy_worker (GH#4398) jakirkham * Type TaskGroup in active_states (GH#4408) jakirkham * Fix test_as_current_is_thread_local (GH#4402) jakirkham * Use list comprehensions to bind TaskGroup type (GH#4401) jakirkham * Make tests pass after 2028 (GH#4403) Bernhard M. Wiedemann * Fix compilation warnings, decide_worker now a C func, stealing improvements (GH#4375) jakirkham * Drop custom __eq__ from Status (GH#4270) jakirkham * test_performance_report: skip without bokeh (GH#4388) Bruno Pagani * Nanny now respects dask settings from ctx mgr (GH#4378) Florian Jetter * Better task duration estimates for outliers (GH#4213) selshowk * Dask internal inherit config (GH#4364) Jacob Tomlinson * Provide setup.py option to profile Cython code (GH#4362) jakirkham * Optimizations of *State and Task* objects and stealing (GH#4358) jakirkham * Cast SortedDict s to dict s in a few key places & other minor changes (GH#4355) jakirkham * Use task annotation priorities for user-level priorities (GH#4354) James Bourbeau * Added docs to highlevelgraph pack/unpack (GH#4352) Mads R. B. Kristensen * Optimizations in notable functions used by transitions (GH#4351) jakirkham * Silence exception when releasing futures on process shutdown (GH#4309) Benjamin Zaitlen - Deboard blind passenger requirement: scikit-build is only available for Python > 3.6 (because of SciPy) and not needed here ------------------------------------------------------------------- Sat Jan 9 23:23:31 UTC 2021 - Benjamin Greiner - update to version 2020.12.0 * Switched to CalVer for versioning scheme. * The scheduler can now receives Dask HighLevelGraph s instead of raw dictionary task graphs. This allows for a much more efficient communication of task graphs from the client to the scheduler. * Added support for using custom Layer-level annotations like priority, retries, etc. with the dask.annotations context manager. * Updated minimum supported version of Dask to 2020.12.0. * Added many type annotations and updates to allow for gradually Cythonizing the scheduler. - Full changelog on https://distributed.readthedocs.io/en/latest/changelog.html - Cythonize scheduler: is platform dependent package now - Iterative deselection of failing tests: kept adding tests to pytest -k parameter until the test suite did not fail anymore. * pytest-dev/pytest-asyncio#168, pytest-dev/pytest-asyncio#188 still not resolved * upstream just pinned pytest-asyncio gh#dask/distributed#4212 ------------------------------------------------------------------- Thu Dec 24 01:14:13 UTC 2020 - Benjamin Greiner - Support multiple python3 flavors gh#openSUSE/python-rpm-macros#66 - Move test into multibuild in order to avoid dependency cycle with dask-distributed ------------------------------------------------------------------- Thu Dec 3 12:06:24 UTC 2020 - Markéta Machová - Launch tests * around 70 of them are inconsistently (sometimes they pass, sometimes fail) broken by new pytest-asyncio (and similar number of other tests got deselected mostly because of similar names) ------------------------------------------------------------------- Sat Nov 21 22:35:43 UTC 2020 - Arun Persaud - update to version 2.30.1: * Pin pytest-asyncio version (GH#4212) James Bourbeau * Replace AsyncProcess exit handler by weakref.finalize (GH#4184) Peter Andreas Entschev * Remove hard coded connect handshake timeouts (GH#4176) Florian Jetter ------------------------------------------------------------------- Sat Oct 10 19:04:32 UTC 2020 - Arun Persaud - update to version 2.30.0: * Support SubgraphCallable in str_graph() (GH#4148) Mads R. B. Kristensen * Handle exceptions in BatchedSend (GH#4135) Tom Augspurger * Fix for missing : in autosummary docs (GH#4143) Gil Forsyth * Limit GPU metrics to visible devices only (GH#3810) Jacob Tomlinson ------------------------------------------------------------------- Mon Oct 5 20:16:12 UTC 2020 - Arun Persaud - update to version 2.29.0: * Use pandas.testing (GH#4138) jakirkham * Fix a few typos (GH#4131) Pav A * Return right away in Cluster.close if cluster is already closed (GH#4116) Tom Rochette * Update async doc with example on .compute() vs client.compute() (GH#4137) Benjamin Zaitlen * Correctly tear down LoopRunner in Client (GH#4112) Sergey Kozlov * Simplify Client._graph_to_futures() (GH#4127) Mads R. B. Kristensen * Cleanup new exception traceback (GH#4125) Krishan Bhasin * Stop writing config files by default (GH#4123) Matthew Rocklin - changes from version 2.28.0: * Fix SSL connection_args for progressbar connect (GH#4122) jennalc ------------------------------------------------------------------- Sat Sep 19 15:08:52 UTC 2020 - Arun Persaud - update to version 2.27.0: * Fix registering a worker plugin with name arg (GH#4105) Nick Evans * Support different remote_python paths on cluster nodes (GH#4085) Abdulelah Bin Mahfoodh * Allow RuntimeError s when closing global clients (GH#4115) Matthew Rocklin * Match pre-commit in dask (GH#4049) Julia Signell * Update super usage (GH#4110) Poruri Sai Rahul ------------------------------------------------------------------- Sat Sep 12 19:58:20 UTC 2020 - Arun Persaud - update to version 2.26.0: * Add logging for adaptive start and stop (GH#4101) Matthew Rocklin * Don’t close a nannied worker if it hasn’t yet started (GH#4093) Matthew Rocklin * Respect timeouts when closing clients synchronously (GH#4096) Matthew Rocklin * Log when downloading a preload script (GH#4094) Matthew Rocklin * dask-worker --nprocs accepts negative values (GH#4089) Dror Speiser * Support zero-worker clients (GH#4090) Matthew Rocklin * Exclude fire-and-forget client from metrics (GH#4078) Tom Augspurger * Drop Serialized.deserialize() method (GH#4073) jakirkham * Add timeout= keyword to Client.wait_for_workers method (GH#4087) Matthew Rocklin ------------------------------------------------------------------- Sat Aug 29 15:53:08 UTC 2020 - Arun Persaud - update to version 2.25.0: * Update for black (GH#4081) Tom Augspurger * Provide informative error when connecting an older version of Dask (GH#4076) Matthew Rocklin * Simplify pack_frames (GH#4068) jakirkham * Simplify frame_split_size (GH#4067) jakirkham * Use list.insert to add prelude up front (GH#4066) jakirkham * Graph helper text (GH#4064) Julia Signell * Graph dashboard: Reset container data if task number is too large (GH#4056) Florian Jetter * Ensure semaphore picks correct IOLoop for threadpool workers (GH#4060) Florian Jetter * Add cluster log method (GH#4051) Jacob Tomlinson * Cleanup more exception tracebacks (GH#4054) Krishan Bhasin * Improve documentation of scheduler.locks options (GH#4062) Florian Jetter - changes from version 2.24.0: * Move toolbar to above and fix y axis (#4043) Julia Signell * Make behavior clearer for how to get worker dashboard (#4047) Julia Signell * Worker dashboard clean up (#4046) Julia Signell * Add a default argument to the datasets and a possibility to override datasets (#4052) Nils Braun * Discover HTTP endpoints (#3744) Martin Durant ------------------------------------------------------------------- Sat Aug 15 17:00:33 UTC 2020 - Arun Persaud - update to version 2.23.0: * Tidy up exception traceback in TCP Comms (GH#4042) Krishan Bhasin * Angle on the x-axis labels (GH#4030) Mathieu Dugré * Always set RMM’s strides in the header (GH#4039) jakirkham * Fix documentation upload_file (GH#4038) Roberto Panai * Update UCX tests for new handshake step (GH#4036) jakirkham * Add test for informative errors in serialization cases (GH#4029) Matthew Rocklin * Add compression, pickle protocol to comm contexts (GH#4019) Matthew Rocklin * Make GPU plots robust to not having GPUs (GH#4008) Matthew Rocklin * Update PendingDeprecationWarning with correct version number (GH#4025) Matthias Bussonnier * Install PyTorch on CI (GH#4017) jakirkham * Try getting cluster dashboard_link before asking scheduler (GH#4018) Matthew Rocklin * Ignore writeable frames with builtin array (GH#4016) jakirkham * Just extend frames2 by frames (GH#4015) jakirkham * Serialize builtin array (GH#4013) jakirkham * Use cuDF’s assert_eq (GH#4014) jakirkham * Clear function cache whenever we upload a new file (GH#3993) Jack Xiaosong Xu * Emmit warning when assign/comparing string with Status Enum (GH#3875) Matthias Bussonnier * Track mutable frames (GH#4004) jakirkham * Improve bytes and bytearray serialization (GH#4009) jakirkham * Fix memory histogram values in dashboard (GH#4006) Willi Rath ------------------------------------------------------------------- Sat Aug 1 22:12:29 UTC 2020 - Arun Persaud - update to version 2.22.0: * Only call frame_split_size when there are frames (GH#3996) jakirkham * Fix failing test_bandwidth (GH#3999) jakirkham * Handle sum of memory percentage when memory_limit is 0 (GH#3984) Julia Signell * Drop msgpack pre-0.5.2 compat code (GH#3977) jakirkham * Revert to localhost for local IP if no network available (GH#3991) Matthew Rocklin * Add missing backtick in inline directive. (GH#3988) Matthias Bussonnier * Warn when threads_per_worker is set to zero (GH#3986) Julia Signell * Use memoryview in unpack_frames (GH#3980) jakirkham * Iterate over list of comms (GH#3959) Matthew Rocklin * Streamline pack_frames/unpack_frames frames (GH#3973) jakirkham * Always attempt to create dask-worker-space folder and continue if it exists (GH#3972) Jendrik Jördening * Use merge_frames with host memory only (GH#3971) jakirkham * Simplify pack_frames_prelude (GH#3961) jakirkham * Use continuation prompt for proper example parsing (GH#3966) Matthias Bussonnier * Ensure writable frames (GH#3967) jakirkham ------------------------------------------------------------------- Sat Jul 18 18:13:10 UTC 2020 - Arun Persaud - update to version 2.21.0: * Fix data replication error (GH#3963) Andrew Fulton * Treat falsey local directory as None (GH#3964) Tom Augspurger * Unpin numpydoc now that 1.1 is released (GH#3957) Gil Forsyth * Error hard when Dask has mismatched versions or lz4 installed (GH#3936) Matthew Rocklin * Skip coercing to bytes in merge_frames (GH#3960) jakirkham * UCX: reuse endpoints in order to fix NVLINK issue (GH#3953) Mads R. B. Kristensen * Optionally use pickle5 (GH#3849) jakirkham * Update time per task chart with filtering and pie (GH#3933) Benjamin Zaitlen * UCX: explicit shutdown message (GH#3950) Mads R. B. Kristensen * Avoid too aggressive retry of connections (GH#3944) Matthias Bussonnier * Parse timeouts in Client.sync (GH#3952) Matthew Rocklin * Synchronize on non-trivial CUDA frame transmission (GH#3949) jakirkham * Serialize memoryview with shape and format (GH#3947) jakirkham * Move scheduler_comm into Cluster.__init__ (GH#3945) Matthew Rocklin ------------------------------------------------------------------- Thu Jul 9 12:28:22 UTC 2020 - Marketa Calabkova - update to 2.20.0 * Link issue on using ``async`` with ``executor_submit`` (:pr:`3939`) `jakirkham`_ * Make dashboard server listens on all IPs by default even when interface is set explicitly (:pr:`3941`) `Loïc Estève`_ * Update logic for worker removal in check ttl (:pr:`3927`) `Benjamin Zaitlen`_ * Close a created cluster quietly (:pr:`3935`) `Matthew Rocklin`_ * Ensure ``Worker.run*`` handles ``kwargs`` correctly (:pr:`3937`) `jakirkham`_ * Restore ``Scheduler.time_started`` for Dask Gateway (:pr:`3934`) `Tom Augspurger`_ * Fix exception handling in ``_wait_until_connected`` (:pr:`3912`) `Alexander Clausen`_ * Make local directory if it does not exist (:pr:`3928`) `Matthew Rocklin`_ * Install vanilla status route if bokeh dependency is not satisfied (:pr:`3844`) `joshreback`_ * Make ``Worker.delete_data`` sync (:pr:`3922`) `Peter Andreas Entschev`_ * Fix ``ensure_bytes`` import location (:pr:`3919`) `jakirkham`_ * Fix race condition in repeated calls to ``cluster.adapt()`` (:pr:`3915`) `Jacob Tomlinson`_ 2.19.0 * Notify worker plugins when a task is released (:pr:`3817`) `Nick Evans`_ * Update heartbeat checks in scheduler (:pr:`3896`) `Benjamin Zaitlen`_ * Make encryption default if ``Security`` is given arguments (:pr:`3887`) `Matthew Rocklin`_ * Show ``cpu_fraction`` on hover for dashboard workers circle plot. (:pr:`3906`) `Loïc Estève`_ * Prune virtual client on variable deletion (:pr:`3910`) `Marco Neumann`_ * Fix total aggregated metrics in dashboard (:pr:`3897`) `Loïc Estève`_ * Support Bokeh 2.1 (:pr:`3904`) `Matthew Rocklin`_ * Update ``related-work.rst`` (:pr:`3889`) `DomHudson`_ * Skip ``test_pid_file`` in older versions of Python (:pr:`3888`) `Matthew Rocklin`_ * Replace ``stream=`` with ``comm=`` in handlers (:pr:`3860`) `Julien Jerphanion`_ * Check hosts for ``None`` value in SSH cluster. (:pr:`3883`) `Matthias Bussonnier`_ * Allow dictionaries in ``security=`` keywords (:pr:`3874`) `Matthew Rocklin`_ * Use pickle protocol 5 with NumPy object arrays (:pr:`3871`) `jakirkham`_ * Cast any ``frame`` to ``uint8`` (same type as ``bytes``) (:pr:`3870`) `jakirkham`_ * Use ``Enum`` for worker, scheduler and nanny status. (:pr:`3853`) `Matthias Bussonnier`_ * Drop legacy ``buffer_interface`` assignment (:pr:`3869`) `jakirkham`_ * Drop old frame splitting in NumPy serialization (:pr:`3868`) `jakirkham`_ * Drop no longer needed local ``import pickle`` (:pr:`3865`) `jakirkham`_ * Fix typo in ``feed``'s log message (:pr:`3867`) `jakirkham`_ * Tidy pickle (:pr:`3866`) `jakirkham`_ * Handle empty times in task stream (:pr:`3862`) `Benjamin Zaitlen`_ * Change ``asyncssh`` objects to sphinx references (:pr:`3861`) `Jacob Tomlinson`_ * Improve ``SSHCluster`` docstring for ``connect_options`` (:pr:`3859`) `Jacob Tomlinson`_ * Validate address parameter in client constructor (:pr:`3842`) `joshreback`_ * Use ``SpecCluster`` name in worker names (:pr:`3855`) `Loïc Estève`_ * Allow async ``add_worker`` and ``remove_worker`` plugin methods (:pr:`3847`) `James Bourbeau`_ 2.18.0 * Merge frames in ``deserialize_bytes`` (:pr:`3639`) `John Kirkham`_ * Allow ``SSHCluster`` to take a list of ``connect_options`` (:pr:`3854`) `Jacob Tomlinson`_ * Add favicon to performance report (:pr:`3852`) `Jacob Tomlinson`_ * Add dashboard plots for the amount of time spent per key and for transfer/serialization (:pr:`3792`) `Benjamin Zaitlen`_ * Fix variable name in journey of a task documentation (:pr:`3840`) `Matthias Bussonnier`_ * Fix typo in journey of a task doc (:pr:`3838`) `James Bourbeau`_ * Register ``dask_cudf`` serializers (:pr:`3832`) `John Kirkham`_ * Fix key check in ``rebalance`` missing keys (:pr:`3834`) `Jacob Tomlinson`_ * Allow collection of partial profile information in case of exceptions (:pr:`3773`) `Florian Jetter`_ 2.17.0 * Record the time since the last run task on the scheduler (:pr:`3830`) `Matthew Rocklin`_ * Set colour of ``nbytes`` pane based on thresholds (:pr:`3805`) `Krishan Bhasin`_ * Include total number of tasks in the performance report (:pr:`3822`) `Abdulelah Bin Mahfoodh`_ * Allow to pass in task key strings in the worker restrictions (:pr:`3826`) `Nils Braun`_ * Control de/ser offload (:pr:`3793`) `Martin Durant`_ * Parse timeout parameters in ``Variable``/``Event``/``Lock`` to support text timeouts (:pr:`3825`) `Nils Braun`_ * Don't send empty dependencies (:pr:`3423`) `Jakub Beránek`_ * Add distributed Dask ``Event`` that mimics ``threading.Event`` (:pr:`3821`) `Nils Braun`_ * Enhance ``VersionMismatchWarning`` messages (:pr:`3786`) `Abdulelah Bin Mahfoodh`_ * Support Pickle's protocol 5 (:pr:`3784`) `jakirkham`_ * Replace ``utils.ignoring`` with ``contextlib.suppress`` (:pr:`3819`) `Nils Braun`_ * Make re-creating conda environments from the CI output easier (:pr:`3816`) `Lucas Rademaker`_ * Add prometheus metrics for semaphore (:pr:`3757`) `Lucas Rademaker`_ * Fix worker plugin called with superseded transition (:pr:`3812`) `Nick Evans`_ * Add retries to server listen (:pr:`3801`) `Jacob Tomlinson`_ * Remove commented out lines from ``scheduler.py`` (:pr:`3803`) `James Bourbeau`_ * Fix ``RuntimeWarning`` for never awaited coroutine when using ``distributed.Semaphore`` (:pr:`3713`) `Florian Jetter`_ * Fix profile thread leakage during test teardown on some platforms (:pr:`3795`) `Florian Jetter`_ * Await self before handling comms (:pr:`3788`) `Matthew Rocklin`_ * Fix typo in ``Cluster`` docstring (:pr:`3787`) `Scott Sanderson`_ 2.16.0 * ``Client.get_dataset`` to always create ``Futures`` attached to itself (:pr:`3729`) `crusaderky`_ * Remove dev-requirements since it is unused (:pr:`3782`) `Julia Signell`_ * Use bokeh column for ``/system`` instead of custom css (:pr:`3781`) `Julia Signell`_ * Attempt to fix ``test_preload_remote_module`` on windows (:pr:`3775`) `James Bourbeau`_ * Fix broadcast for TLS comms (:pr:`3766`) `Florian Jetter`_ * Don't validate http preloads locally (:pr:`3768`) `Rami Chowdhury`_ * Allow range of ports to be specified for ``Workers`` (:pr:`3704`) `James Bourbeau`_ * Add UCX support for RDMACM (:pr:`3759`) `Peter Andreas Entschev`_ * Support web addresses in preload (:pr:`3755`) `Matthew Rocklin`_ ------------------------------------------------------------------- Sat May 2 20:27:24 UTC 2020 - Arun Persaud - update to version 2.15.2: * Connect to dashboard when address provided (GH#3758) Tom Augspurger * Move test_gpu_metrics test (GH#3721) Tom Augspurger * Nanny closing worker on KeyboardInterrupt (GH#3747) Mads R. B. Kristensen * Replace OrderedDict with dict in scheduler (GH#3740) Matthew Rocklin * Fix exception handling typo (GH#3751) Jonas Haag - changes from version 2.15.1: * Ensure BokehTornado uses prefix (GH#3746) James Bourbeau * Warn if cluster closes before starting (GH#3735) Matthew Rocklin * Memoryview serialisation (GH#3743) Martin Durant * Allows logging config under distributed key (GH#2952) Dillon Niederhut ------------------------------------------------------------------- Sat Apr 25 16:23:42 UTC 2020 - Arun Persaud - update to version 2.15.0: * Reinstate support for legacy @gen_cluster functions (GH#3738) crusaderky * Relax NumPy requirement in UCX (GH#3731) jakirkham * Add Configuration Schema (GH#3696) Matthew Rocklin * Reuse CI scripts for local installation process (GH#3698) crusaderky * Use PeriodicCallback class from tornado (GH#3725) James Bourbeau * Add remote_python option in ssh cmd (GH#3709) Abdulelah Bin Mahfoodh * Configurable polling interval for cluster widget (GH#3723) Julia Signell * Fix copy-paste in docs (GH#3728) Julia Signell * Replace gen.coroutine with async-await in tests (GH#3706) crusaderky * Fix flaky test_oversubscribing_leases (GH#3726) Florian Jetter * Add batch_size to Client.map (GH#3650) Tom Augspurger * Adjust semaphore test timeouts (GH#3720) Florian Jetter * Dask-serialize dicts longer than five elements (GH#3689) Richard J Zamora * Force threads_per_worker (GH#3715) crusaderky * Idempotent semaphore acquire with retries (GH#3690) Florian Jetter * Always use readinto in TCP (GH#3711) jakirkham * Avoid DeprecationWarning from pandas (GH#3712) Tom Augspurger * Allow modification of distributed.comm.retry at runtime (GH#3705) Florian Jetter * Do not log an error on unset variable delete (GH#3652) Jonathan J. Helmus * Add remote_python keyword to the new SSHCluster (GH#3701) Abdulelah Bin Mahfoodh * Replace Example with Examples in docstrings (GH#3697) Matthew Rocklin * Add Cluster __enter__ and __exit__ methods (GH#3699) Matthew Rocklin * Fix propagating inherit config in SSHCluster for non-bash shells (GH#3688) Abdulelah Bin Mahfoodh * Add Client.wait_to_workers to Client autosummary table (GH#3692) James Bourbeau * Replace Bokeh Server with Tornado HTTPServer (GH#3658) Matthew Rocklin * Fix dask-ssh after removing local-directory from dask_scheduler cli (GH#3684) Abdulelah Bin Mahfoodh * Support preload modules in Nanny (GH#3678) Matthew Rocklin * Refactor semaphore internals: make _get_lease synchronous (GH#3679) Lucas Rademaker * Don’t make task graphs too big (GH#3671) Martin Durant * Pass through connection/listen_args as splatted keywords (GH#3674) Matthew Rocklin * Run preload at import, start, and teardown (GH#3673) Matthew Rocklin * Use relative URL in scheduler dashboard (GH#3676) Nicholas Smith * Expose Security object as public API (GH#3675) Matthew Rocklin * Add zoom tools to profile plots (GH#3672) James Bourbeau * Update Scheduler.rebalance return value when data is missing (GH#3670) James Bourbeau ------------------------------------------------------------------- Thu Apr 23 11:08:38 UTC 2020 - Tomáš Chvátal - Remove py2 deps as the package is py3 only ------------------------------------------------------------------- Sat Apr 11 21:46:23 UTC 2020 - Arun Persaud - update to version 2.14.0: * Enable more UCX tests (GH#3667) jakirkham * Remove openssl 1.1.1d pin for Travis (GH#3668) Jonathan J. Helmus * More documentation for Semaphore (GH#3664) Florian Jetter * Get CUDA context to finalize Numba DeviceNDArray (GH#3666) jakirkham * Add Resouces option to get_task_stream and call output_file (GH#3653) Prasun Anand * Add Semaphore extension (GH#3573) Lucas Rademaker * Replace ncores with nthreads in work stealing tests (GH#3615) James Bourbeau * Clean up some test warnings (GH#3662) Matthew Rocklin * Write “why killed” docs (GH#3596) Martin Durant * Update Python version checking (GH#3660) James Bourbeau * Add newlines to ensure code formatting for retire_workers (GH#3661) Rami Chowdhury * Clean up performance report test (GH#3655) Matthew Rocklin * Avoid diagnostics time in performance report (GH#3654) Matthew Rocklin * Introduce config for default task duration (GH#3642) Gabriel Sailer * UCX simplify receiving frames in comm (GH#3651) jakirkham * Bump checkout GitHub action to v2 (GH#3649) James Bourbeau * Handle exception in faulthandler (GH#3646) Jacob Tomlinson * Add prometheus metric for suspicious tasks (GH#3550) Gabriel Sailer * Remove local-directory keyword (GH#3620) Prasun Anand * Don’t create output Futures in Client when there are mixed Client Futures (GH#3643) James Bourbeau * Add link to contributing.md (GH#3621) Prasun Anand * Update bokeh dependency in CI builds (GH#3637) James Bourbeau ------------------------------------------------------------------- Sat Mar 28 16:48:48 UTC 2020 - Arun Persaud - update to version 2.13.0: * UCX synchronize default stream only on CUDA frames (GH#3638) Peter Andreas Entschev * Add as_completed.clear method (GH#3617) Matthew Rocklin * Drop unused line from pack_frames_prelude (GH#3634) John Kirkham * Add logging message when closing idle dask scheduler (GH#3632) Matthew Rocklin * Include frame lengths of CUDA objects in header["lengths"] (GH#3631) John Kirkham * Ensure Client connection pool semaphore attaches to the Client event loop (GH#3546) James Bourbeau * Remove dead stealing code (GH#3619) Florian Jetter * Check nbytes and types before reading data (GH#3628) John Kirkham * Ensure that we don’t steal blacklisted fast tasks (GH#3591) Florian Jetter * Support async Listener.stop functions (GH#3613) Matthew Rocklin * Add str/repr methods to as_completed (GH#3618) Matthew Rocklin * Add backoff to comm connect attempts. (GH#3496) Matthias Urlichs * Make Listeners awaitable (GH#3611) Matthew Rocklin * Increase number of visible mantissas in dashboard plots (GH#3585) Scott Sievert * Pin openssl to 1.1.1d for Travis (GH#3602) Jacob Tomlinson * Replace tornado.queues with asyncio.queues (GH#3607) James Bourbeau * Remove dill from CI environments (GH#3608) Loïc Estève * Fix linting errors (GH#3604) James Bourbeau * Synchronize default CUDA stream before UCX send/recv (GH#3598) Peter Andreas Entschev * Add configuration for Adaptive arguments (GH#3509) Gabriel Sailer * Change Adaptive docs to reference adaptive_target (GH#3597) Julia Signell * Optionally compress on a frame-by-frame basis (GH#3586) Matthew Rocklin * Add Python version to version check (GH#3567) James Bourbeau * Import tlz (GH#3579) John Kirkham * Pin numpydoc to avoid double escaped * (GH#3530) Gil Forsyth * Avoid performance_report crashing when a worker dies mid-compute (GH#3575) Krishan Bhasin * Pin bokeh in CI builds (GH#3570) James Bourbeau * Disable fast fail on GitHub Actions Windows CI (GH#3569) James Bourbeau * Fix typo in Client.shutdown docstring (GH#3562) John Kirkham * Add local_directory option to dask-ssh (GH#3554) Abdulelah Bin Mahfoodh ------------------------------------------------------------------- Sun Mar 8 19:04:45 UTC 2020 - Arun Persaud - update to version 2.12.0: * Update TaskGroup remove logic (GH#3557) James Bourbeau * Fix-up CuPy sparse serialization (GH#3556) John Kirkham * API docs for LocalCluster and SpecCluster (GH#3548) Tom Augspurger * Serialize sparse arrays (GH#3545) John Kirkham * Allow tasks with restrictions to be stolen (GH#3069) Stan Seibert * Use UCX default configuration instead of raising (GH#3544) Peter Andreas Entschev * Support using other serializers with register_generic (GH#3536) John Kirkham * DOC: update to async await (GH#3543) Tom Augspurger * Use pytest.raises in test_ucx_config.py (GH#3541) John Kirkham * Fix/more ucx config options (GH#3539) Benjamin Zaitlen * Update heartbeat CommClosedError error handling (GH#3529) James Bourbeau * Use makedirs when constructing local_directory (GH#3538) John Kirkham * Mark None as MessagePack serializable (GH#3537) John Kirkham * Mark bool as MessagePack serializable (GH#3535) John Kirkham * Use ‘temporary-directory’ from dask.config for Nanny’s directory (GH#3531) John Kirkham * Add try-except around getting source code in performance report (GH#3505) Matthew Rocklin * Fix typo in docstring (GH#3528) Davis Bennett * Make work stealing callback time configurable (GH#3523) Lucas Rademaker * RMM/UCX Config Flags (GH#3515) Benjamin Zaitlen * Revise develop-docs: conda env example (GH#3406) Darren Weber * Remove import ucp from the top of ucx.py (GH#3510) Peter Andreas Entschev * Rename logs to get_logs (GH#3473) Jacob Tomlinson * Stop keep alives when worker reconnecting to the scheduler (GH#3493) Jacob Tomlinson ------------------------------------------------------------------- Sat Feb 22 18:57:41 UTC 2020 - Arun Persaud - update to version 2.11.0: * Add dask serialization of CUDA objects (GH#3482) John Kirkham * Suppress cuML ImportError (GH#3499) John Kirkham * Msgpack 1.0 compatibility (GH#3494) James Bourbeau * Register cuML serializers (GH#3485) John Kirkham * Check exact equality for worker state (GH#3483) Brett Naul * Serialize 1-D, contiguous, uint8 CUDA frames (GH#3475) John Kirkham * Update NumPy array serialization to handle non-contiguous slices (GH#3474) James Bourbeau * Propose fix for collection based resources docs (GH#3480) Chris Roat * Remove --verbose flag from CI runs (GH#3484) Matthew Rocklin * Do not duplicate messages in scheduler report (GH#3477) Jakub Beránek * Register Dask cuDF serializers (GH#3478) John Kirkham * Add support for Python 3.8 (GH#3249) James Bourbeau * Add last seen column to worker table and highlight errant workers (GH#3468) kaelgreco * Change default value of local_directory from empty string to None (GH#3441) condoratberlin * Clear old docs (GH#3458) Matthew Rocklin * Change default multiprocessing behavior to spawn (GH#3461) Matthew Rocklin * Split dashboard host on additional slashes to handle inproc (GH#3466) Jacob Tomlinson * Update locality.rst (GH#3470) Dustin Tindall * Minor gen.Return cleanup (GH#3469) James Bourbeau * Update comparison logic for worker state (GH#3321) rockwellw * Update minimum tblib version to 1.6.0 (GH#3451) James Bourbeau * Add total row to workers plot in dashboard (GH#3464) Julia Signell * Workaround RecursionError on profile data (GH#3455) Tom Augspurger * Include code and summary in performance report (GH#3462) Matthew Rocklin * Skip test_open_close_many_workers on Python 3.6 (GH#3459) Matthew Rocklin * Support serializing/deserializing rmm.DeviceBuffer s (GH#3442) John Kirkham * Always add new TaskGroup to TaskPrefix (GH#3322) James Bourbeau * Rerun black on the code base (GH#3444) John Kirkham * Ensure __causes__ s of exceptions raised on workers are serialized (GH#3430) Alex Adamson * Adjust numba.cuda import and add check (GH#3446) John Kirkham * Fix name of Numba serialization test (GH#3447) John Kirkham * Checks for command parameters in ssh2 (GH#3078) Peter Andreas Entschev * Update worker_kwargs description in LocalCluster constructor (GH#3438) James Bourbeau * Ensure scheduler updates task and worker states after successful worker data deletion (GH#3401) James Bourbeau * Avoid loop= keyword in asyncio coordination primitives (GH#3437) Matthew Rocklin * Call pip as a module to avoid warnings (GH#3436) Cyril Shcherbin * Add documentation of parameters in coordination primitives (GH#3434) Søren Fuglede Jørgensen * Replace tornado.locks with asyncio for Events/Locks/Conditions/Semaphore (GH#3397) Matthew Rocklin * Remove object from class hierarchy (GH#3432) Anderson Banihirwe * Add dashboard_link property to Client (GH#3429) Jacob Tomlinson * Allow memory monitor to evict data more aggressively (GH#3424) fjetter * Make _get_ip return an IP address when defaulting (GH#3418) Pierre Glaser * Support version checking with older versions of Dask (GH#3390) Igor Gotlibovych * Add Mac OS build to CI (GH#3358) James Bourbeau ------------------------------------------------------------------- Sat Feb 8 21:46:57 UTC 2020 - Arun Persaud - update to version 2.10.0: * Fixed ZeroDivisionError in dashboard when no workers were present (GH#3407) James Bourbeau * Respect the dashboard-prefix when redirecting from the root (GH#3387) Chrysostomos Nanakos * Allow enabling / disabling work-stealing after the cluster has started (GH#3410) John Kirkham * Support *args and **kwargs in offload (GH#3392) Matthew Rocklin * Add lifecycle hooks to SchedulerPlugin (GH#3391) Matthew Rocklin ------------------------------------------------------------------- Sat Jan 18 19:17:28 UTC 2020 - Arun Persaud - update to version 2.9.3: * Raise RuntimeError if no running loop (GH#3385) James Bourbeau * Fix get_running_loop import (GH#3383) James Bourbeau * Get JavaScript document location instead of window and handle proxied url (GH#3382) Jacob Tomlinson - changes from version 2.9.2: * Move Windows CI to GitHub Actions (GH#3373) Jacob Tomlinson * Add client join and leave hooks (GH#3371) Jacob Tomlinson * Add cluster map dashboard (GH#3361) Jacob Tomlinson * Close connection comm on retry (GH#3365) James Bourbeau * Fix scheduler state in case of worker name collision (GH#3366) byjott * Add --worker-class option to dask-worker CLI (GH#3364) James Bourbeau * Remove locale check that fails on OS X (GH#3360) Jacob Tomlinson * Rework version checking (GH#2627) Matthew Rocklin * Add websocket scheduler plugin (GH#3335) Jacob Tomlinson * Return task in dask-worker on_signal function (GH#3354) James Bourbeau * Fix failures on mixed integer/string worker names (GH#3352) Benedikt Reinartz * Avoid calling nbytes multiple times when sending data (GH#3349) Markus Mohrhard * Avoid setting event loop policy if within IPython kernel and no running event loop (GH#3336) Mana Borwornpadungkitti * Relax intermittent failing test_profile_server (GH#3346) Matthew Rocklin ------------------------------------------------------------------- Sat Jan 11 18:24:51 UTC 2020 - Arun Persaud - specfile: * update copyright year - update to version 2.9.1: * Add lock around dumps_function cache (GH#3337) Matthew Rocklin * Add setuptools to dependencies (GH#3320) James Bourbeau * Use TaskPrefix.name in Graph layout (GH#3328) Matthew Rocklin * Add missing “ in performance report example (GH#3329) John Kirkham * Add performance report docs and color definitions to docs (GH#3325) Benjamin Zaitlen * Switch startstops to dicts and add worker name to transfer (GH#3319) Jacob Tomlinson * Add plugin entry point for out-of-tree comms library (GH#3305) Patrick Sodré * All scheduler task states in prometheus (GH#3307) fjetter * Use worker name in logs (GH#3309) Stephan Erb * Add TaskGroup and TaskPrefix scheduler state (GH#3262) Matthew Rocklin * Update latencies with heartbeats (GH#3310) fjetter * Update inlining Futures in task graph in Client._graph_to_futures (GH#3303) James Bourbeau * Use hostname as default IP address rather than localhost (GH#3308) Matthew Rocklin * Clean up flaky test_nanny_throttle (GH#3295) Tom Augspurger * Add lock to scheduler for sensitive operations (GH#3259) Matthew Rocklin * Log address for each of the Scheduler listerners (GH#3306) Matthew Rocklin * Make ConnectionPool.close asynchronous (GH#3304) Matthew Rocklin ------------------------------------------------------------------- Sat Dec 7 19:09:30 UTC 2019 - Arun Persaud - specfile: * removed dask-submit, dask-remote - update to version 2.9.0: * Add dask-spec CLI tool (GH#3090) Matthew Rocklin * Connectionpool: don’t hand out closed connections (GH#3301) byjott * Retry operations on network issues (GH#3294) byjott * Skip Security.temporary() tests if cryptography not installed (GH#3302) James Bourbeau * Support multiple listeners in the scheduler (GH#3288) Matthew Rocklin * Updates RMM comment to the correct release (GH#3299) John Kirkham * Add title to performance_report (GH#3298) Matthew Rocklin * Forgot to fix slow test (GH#3297) Benjamin Zaitlen * Update SSHCluster docstring parameters (GH#3296) James Bourbeau * worker.close() awaits batched_stream.close() (GH#3291) Mads R. B. Kristensen * Fix asynchronous listener in UCX (GH#3292) Benjamin Zaitlen * Avoid repeatedly adding deps to already in memory stack (GH#3293) James Bourbeau * xfail ucx empty object typed dataframe (GH#3279) Benjamin Zaitlen * Fix distributed.wait documentation (GH#3289) Tom Rochette * Move Python 3 syntax tests into main tests (GH#3281) Matthew Rocklin * xfail test_workspace_concurrency for Python 3.6 (GH#3283) Matthew Rocklin * Add performance_report context manager for static report generation (GH#3282) Matthew Rocklin * Update function serialization caches with custom LRU class (GH#3260) James Bourbeau * Make Listener.start asynchronous (GH#3278) Matthew Rocklin * Remove dask-submit and dask-remote (GH#3280) Matthew Rocklin * Worker profile server (GH#3274) Matthew Rocklin * Improve bandwidth workers plot (GH#3273) Matthew Rocklin * Make profile coroutines consistent between Scheduler and Worker (GH#3277) Matthew Rocklin * Enable saving profile information from server threads (GH#3271) Matthew Rocklin * Remove memory use plot (GH#3269) Matthew Rocklin * Add offload size to configuration (GH#3270) Matthew Rocklin * Fix layout scaling on profile plots (GH#3268) Jacob Tomlinson * Set x_range in CPU plot based on the number of threads (GH#3266) Matthew Rocklin * Use base-2 values for byte-valued axes in dashboard (GH#3267) Matthew Rocklin * Robust gather in case of connection failures (GH#3246) fjetter * Use DeviceBuffer from newer RMM releases (GH#3261) John Kirkham * Fix dev requirements for pytest (GH#3264) Elliott Sales de Andrade * Add validate options to configuration (GH#3258) Matthew Rocklin ------------------------------------------------------------------- Sun Nov 24 17:36:02 UTC 2019 - Arun Persaud - update to version 2.8.1: * Fix hanging worker when the scheduler leaves (GH#3250) Tom Augspurger * Fix NumPy writeable serialization bug (GH#3253) James Bourbeau * Skip numba.cuda tests if CUDA is not available (GH#3255) Peter Andreas Entschev * Add new dashboard plot for memory use by key (GH#3243) Matthew Rocklin * Fix array.shape() -> array.shape (GH#3247) Jed Brown * Fixed typos in pubsub.py (GH#3244) He Jia * Fixed cupy array going out of scope (GH#3240) Mads R. B. Kristensen * Remove gen.coroutine usage in scheduler (GH#3242) Jim Crist-Harif * Use inspect.isawaitable where relevant (GH#3241) Jim Crist-Harif ------------------------------------------------------------------- Sat Nov 16 17:54:33 UTC 2019 - Arun Persaud - specfile: * updated required versions according to requirement.txt - update to version 2.8.0: * Add UCX config values (GH#3135) Matthew Rocklin * Relax test_MultiWorker (GH#3210) Matthew Rocklin * Avoid ucp.init at import time (GH#3211) Matthew Rocklin * Clean up rpc to avoid intermittent test failure (GH#3215) Matthew Rocklin * Respect protocol if given to Scheduler (GH#3212) Matthew Rocklin * Use legend_field= keyword in bokeh plots (GH#3218) Matthew Rocklin * Cache psutil.Process object in Nanny (GH#3207) Matthew Rocklin * Replace gen.sleep with asyncio.sleep (GH#3208) Matthew Rocklin * Avoid offloading serialization for small messages (GH#3224) Matthew Rocklin * Add desired_workers metric (GH#3221) Gabriel Sailer * Fail fast when importing distributed.comm.ucx (GH#3228) Matthew Rocklin * Add module name to Future repr (GH#3231) Matthew Rocklin * Add name to Pub/Sub repr (GH#3235) Matthew Rocklin * Import CPU_COUNT from dask.system (GH#3199) James Bourbeau * Efficiently serialize zero strided NumPy arrays (GH#3180) James Bourbeau * Cache function deserialization in workers (GH#3234) Matthew Rocklin * Respect ordering of futures in futures_of (GH#3236) Matthew Rocklin * Bump dask dependency to 2.7.0 (GH#3237) James Bourbeau * Avoid setting inf x_range (GH#3229) rockwellw * Clear task stream based on recent behavior (GH#3200) Matthew Rocklin * Use the percentage field for profile plots (GH#3238) Matthew Rocklin - changes from version 2.7.0 : * Adds badges to README.rst [skip ci] (GH#3152) James Bourbeau * Don’t overwrite self.address if it is present (GH#3153) Gil Forsyth * Remove outdated references to debug scheduler and worker bokeh pages. (GH#3160) darindf * Update CONTRIBUTING.md (GH#3159) Jacob Tomlinson * Add Prometheus metric for a worker’s executing tasks count (GH#3163) darindf * Update Prometheus documentation (GH#3165) darindf * Fix Numba serialization when strides is None (GH#3166) Peter Andreas Entschev * Await cluster in Adaptive.recommendations (GH#3168) Simon Boothroyd * Support automatic TLS (GH#3164) Jim Crist * Avoid swamping high-memory workers with data requests (GH#3071) Tom Augspurger * Update UCX variables to use sockcm by default (GH#3177) Peter Andreas Entschev * Get protocol in Nanny/Worker from scheduler address (GH#3175) Peter Andreas Entschev * Add worker and tasks state for Prometheus data collection (GH#3174) darindf * Use async def functions for offload to/from_frames (GH#3171) Mads R. B. Kristensen * Subprocesses inherit the global dask config (GH#3192) Mads R. B. Kristensen * XFail test_open_close_many_workers (GH#3194) Matthew Rocklin * Drop Python 3.5 (GH#3179) James Bourbeau * UCX: avoid double init after fork (GH#3178) Mads R. B. Kristensen * Silence warning when importing while offline (GH#3203) James A. Bednar * Adds docs to Client methods for resources, actors, and traverse (GH#2851) IPetrik * Add test for concurrent scatter operations (GH#2244) Matthew Rocklin * Expand async docs (GH#2293) Dave Hirschfeld * Add PatchedDeviceArray to drop stride attribute for cupy<7.0 (GH#3198) Richard J Zamora - changes from version 2.6.0: * Refactor dashboard module (GH#3138) Jacob Tomlinson * Use setuptools.find_packages in setup.py (GH#3150) Matthew Rocklin * Move death timeout logic up to Node.start (GH#3115) Matthew Rocklin * Only include metric in WorkerTable if it is a scalar (GH#3140) Matthew Rocklin * Add Nanny(config={...}) keyword (GH#3134) Matthew Rocklin * Xfail test_worksapce_concurrency on Python 3.6 (GH#3132) Matthew Rocklin * Extend Worker plugin API with transition method (GH#2994) matthieubulte * Raise exception if the user passes in unused keywords to Client (GH#3117) Jonathan De Troye * Move new SSHCluster to top level (GH#3128) Matthew Rocklin * Bump dask dependency (GH#3124) Jim Crist - changes from version 2.5.2: * Make dask-worker close quietly when given sigint signal (GH#3116) Matthew Rocklin * Replace use of tornado.gen with asyncio in dask-worker (GH#3114) Matthew Rocklin * UCX: allocate CUDA arrays using RMM and Numba (GH#3109) Mads R. B. Kristensen * Support calling cluster.scale as async method (GH#3110) Jim Crist * Identify lost workers in SpecCluster based on address not name (GH#3088) James Bourbeau * Add Client.shutdown method (GH#3106) Matthew Rocklin * Collect worker-worker and type bandwidth information (GH#3094) Matthew Rocklin * Send noise over the wire to keep dask-ssh connection alive (GH#3105) Gil Forsyth * Retry scheduler connect multiple times (GH#3104) Jacob Tomlinson * Add favicon of logo to the dashboard (GH#3095) James Bourbeau * Remove utils.py functions for their dask/utils.py equivalents (GH#3042) Matthew Rocklin * Lower default bokeh log level (GH#3087) Philipp Rudiger * Check if self.cluster.scheduler is a local scheduler (GH#3099) Jacob Tomlinson - changes from version 2.5.1: * Support clusters that don’t have .security or ._close methods (GH#3100) Matthew Rocklin - changes from version 2.5.0: * Use the new UCX Python bindings (GH#3059) Mads R. B. Kristensen * Fix worker preload config (GH#3027) byjott * Fix widget with spec that generates multiple workers (GH#3067) Loïc Estève * Make Client.get_versions async friendly (GH#3064) Jacob Tomlinson * Add configuation option for longer error tracebacks (GH#3086) Daniel Farrell * Have Client get Security from passed Cluster (GH#3079) Matthew Rocklin * Respect Cluster.dashboard_link in Client._repr_html_ if it exists (GH#3077) Matthew Rocklin * Add monitoring with dask cluster docs (GH#3072) Arpit Solanki * Protocol of cupy and numba handles serialization exclusively (GH#3047) Mads R. B. Kristensen * Allow specification of worker type in SSHCLuster (GH#3061) Jacob Tomlinson * Use Cluster.scheduler_info for workers= value in repr (GH#3058) Matthew Rocklin * Allow SpecCluster to scale by memory and cores (GH#3057) Matthew Rocklin * Allow full script in preload inputs (GH#3052) Matthew Rocklin * Check multiple cgroups dirs, ceil fractional cpus (GH#3056) Jim Crist * Add blurb about disabling work stealing (GH#3055) Chris White ------------------------------------------------------------------- Sat Sep 14 16:19:32 UTC 2019 - Arun Persaud - specfile: * removed six - update to version 2.4.0: * Remove six (GH#3045) Matthew Rocklin * Add missing test data to sdist tarball (GH#3050) Elliott Sales de Andrade * Use mock from unittest standard library (GH#3049) Elliott Sales de Andrade * Use cgroups resource limits to determine default threads and memory (GH#3039) Jim Crist * Move task deserialization to immediately before task execution (GH#3015) James Bourbeau * Drop joblib shim module in distributed (GH#3040) John Kirkham * Redirect configuration doc page (GH#3038) Matthew Rocklin * Support --name 0 and --nprocs keywords in dask-worker cli (GH#3037) Matthew Rocklin * Remove lost workers from SpecCluster.workers (GH#2990) Guillaume Eynard-Bontemps * Clean up test_local.py::test_defaults (GH#3017) Matthew Rocklin * Replace print statement in Queue.__init__ with debug message (GH#3035) Mikhail Akimov * Set the x_range limit of the Meory utilization plot to memory-limit (GH#3034) Matthew Rocklin * Rely on cudf codebase for cudf serialization (GH#2998) Benjamin Zaitlen * Add fallback html repr for Cluster (GH#3023) Jim Crist * Add support for zstandard compression to comms (GH#2970) Abael He * Avoid collision when using os.environ in dashboard_link (GH#3021) Matthew Rocklin * Fix ConnectionPool limit handling (GH#3005) byjott * Support Spec jobs that generate multiple workers (GH#3013) Matthew Rocklin * Tweak Logs styling (GH#3012) Jim Crist * Better name for cudf deserialization function name (GH#3008) Benjamin Zaitlen * Make spec.ProcessInterface a valid no-op worker (GH#3004) Matthew Rocklin * Return dictionaries from new_worker_spec rather than name/worker pairs (GH#3000) Matthew Rocklin * Fix minor typo in documentation (GH#3002) Mohammad Noor * Permit more keyword options when scaling with cores and memory (GH#2997) Matthew Rocklin * Add cuda_ipc to UCX environment for NVLink (GH#2996) Benjamin Zaitlen * Add threads= and memory= to Cluster and Client reprs (GH#2995) Matthew Rocklin * Fix PyNVML initialization (GH#2993) Richard J Zamora ------------------------------------------------------------------- Sat Aug 31 04:50:18 UTC 2019 - Arun Persaud - update to version 2.3.2: * Skip exceptions in startup information (GH#2991) Jacob Tomlinson - changes from version 2.3.1: * Add support for separate external address for SpecCluster scheduler (GH#2963) Jacob Tomlinson * Defer cudf serialization/deserialization to that library (GH#2881) Benjamin Zaitlen * Workaround for hanging test now calls ucp.fin() (GH#2967) Mads R. B. Kristensen * Remove unnecessary bullet point (GH#2972) Pav A * Directly import progress from diagnostics.progressbar (GH#2975) Matthew Rocklin * Handle buffer protocol objects in ensure_bytes (GH#2969) Tom Augspurger * Fix documentatation syntax and tree (GH#2981) Pav A * Improve get_ip_interface error message when interface does not exist (GH#2964) Loïc Estève * Add cores= and memory= keywords to scale (GH#2974) Matthew Rocklin * Make workers robust to bad custom metrics (GH#2984) Matthew Rocklin - changes from version 2.3.0: * Except all exceptions when checking pynvml (GH#2961) Matthew Rocklin * Pass serialization down through small base collections (GH#2948) Peter Andreas Entschev * Use pytest.warning(Warning) rather than Exception (GH#2958) Matthew Rocklin * Allow server_kwargs to override defaults in dashboard (GH#2955) Bruce Merry * Update utils_perf.py (GH#2954) Shayan Amani * Normalize names with str in retire_workers (GH#2949) Matthew Rocklin * Update client.py (GH#2951) Shayan Amani * Add GPUCurrentLoad dashboard plots (GH#2944) Matthew Rocklin * Pass GPU diagnostics from worker to scheduler (GH#2932) Matthew Rocklin * Import from collections.abc (GH#2938) Jim Crist * Fixes Worker docstring formatting (GH#2939) James Bourbeau * Redirect setup docs to docs.dask.org (GH#2936) Matthew Rocklin * Wrap offload in gen.coroutine (GH#2934) Matthew Rocklin * Change TCP.close to a coroutine to avoid task pending warning (GH#2930) Matthew Rocklin * Fixup black string normalization (GH#2929) Jim Crist * Move core functionality from SpecCluster to Cluster (GH#2913) Matthew Rocklin * Add aenter/aexit protocols to ProcessInterface (GH#2927) Matthew Rocklin * Add real-time CPU utilization plot to dashboard (GH#2922) Matthew Rocklin * Always kill processes in clean tests, even if we don’t check (GH#2924) Matthew Rocklin * Add timeouts to processes in SSH tests (GH#2925) Matthew Rocklin * Add documentation around spec.ProcessInterface (GH#2923) Matthew Rocklin * Cleanup async warnings in tests (GH#2920) Matthew Rocklin * Give 404 when requesting nonexistent tasks or workers (GH#2921) Martin Durant * Raise informative warning when rescheduling an unknown task (GH#2916) James Bourbeau * Fix docstring (GH#2917) Martin Durant * Add keep-alive message between worker and scheduler (GH#2907) Matthew Rocklin * Rewrite Adaptive/SpecCluster to support slowly arriving workers (GH#2904) Matthew Rocklin * Call heartbeat rather than reconnect on disconnection (GH#2906) Matthew Rocklin ------------------------------------------------------------------- Fri Aug 9 14:39:26 UTC 2019 - Marketa Calabkova - Update to version 2.2.0 * Drop support for Python 2 * Lots of other changes, see upstream changelog ------------------------------------------------------------------- Sat Feb 2 17:10:13 UTC 2019 - Arun Persaud - update to version 1.25.3: * Fix excess threading on missing connections (:pr:`2403`) Daniel Farrell * Fix typo in doc (:pr:`2457`) Loïc Estève * Start fewer but larger workers with LocalCluster (:pr:`2452`) Matthew Rocklin * Check for non-zero length first in read loop (:pr:`2465`) John Kirkham * DOC: Use of local cluster in script (:pr:`2462`) Peter Killick * DOC/API: Signature for base class write / read (:pr:`2472`) Tom Augspurger * Support Pytest 4 in Tests (:pr:`2478`) Adam Beberg * Ensure async behavior in event loop with LocalCluster (:pr:`2484`) Matthew Rocklin * Fix spurious CancelledError (:pr:`2485`) Loïc Estève * Properly reset dask.config scheduler and shuffle when closing the client (:pr:`2475`) George Sakkis * Make it more explict that resources are per worker. (:pr:`2470`) Loïc Estève * Remove references to center (:pr:`2488`) Matthew Rocklin * Expand client clearing timeout to 10s in testing (:pr:`2493`) Matthew Rocklin * Propagate key keyword in progressbar (:pr:`2492`) Matthew Rocklin * Use provided cluster's IOLoop if present in Client (:pr:`2494`) Matthew Rocklin ------------------------------------------------------------------- Sun Jan 6 22:33:33 UTC 2019 - Arun Persaud - specfile: * update copyright year - update to version 1.25.2: * Clean up LocalCluster logging better in async mode (:pr:`2448`) Matthew Rocklin * Add short error message if bokeh cannot be imported (:pr:`2444`) Dirk Petersen * Add optional environment variables to Nanny (:pr:`2431`) Matthew Rocklin * Make the direct keyword docstring entries uniform (:pr:`2441`) Matthew Rocklin * Make LocalCluster.close async friendly (:pr:`2437`) Matthew Rocklin * gather_dep: don't request dependencies we already found out we don't want (:pr:`2428`) tjb900 * Add parameters to Client.run docstring (:pr:`2429`) Matthew Rocklin * Support coroutines and async-def functions in run/run_scheduler (:pr:`2427`) Matthew Rocklin * Name threads in ThreadPoolExecutors (:pr:`2408`) Matthew Rocklin ------------------------------------------------------------------- Sat Dec 22 03:59:39 UTC 2018 - Todd R - Update to version 1.25.1 * Serialize numpy.ma.masked objects properly (:pr:`2384`) `Jim Crist`_ * Turn off bokeh property validation in dashboard (:pr:`2387`) `Jim Crist`_ * Fully initialize WorkerState objects (:pr:`2388`) `Jim Crist`_ * Fix typo in scheduler docstring (:pr:`2393`) `Russ Bubley`_ * DOC: fix typo in distributed.worker.Worker docstring (:pr:`2395`) `Loïc Estève`_ * Remove clients and workers from event log after removal (:pr:`2394`) `tjb900`_ * Support msgpack 0.6.0 by providing length keywords (:pr:`2399`) `tjb900`_ * Use async-await on large messages test (:pr:`2404`) `Matthew Rocklin`_ * Fix race condition in normalize_collection (:pr:`2386`) `Jim Crist`_ * Fix redict collection after HighLevelGraph fix upstream (:pr:`2413`) `Matthew Rocklin`_ * Add a blocking argument to Lock.acquire(`) (:pr:`2412`) `Stephan Hoyer`_ * Fix long traceback test (:pr:`2417`) `Matthew Rocklin`_ * Update x509 certificates to current OpenSSL standards. (:pr:`2418`) `Diane Trout`_ ------------------------------------------------------------------- Sat Dec 1 18:37:39 UTC 2018 - Arun Persaud - update to version 1.25.0: * Fixed the 404 error on the Scheduler Dashboard homepage (GH#2361) Michael Wheeler * Consolidate two Worker classes into one (GH#2363) Matthew Rocklin * Avoid warnings in pyarrow and msgpack (GH#2364) Matthew Rocklin * Avoid race condition in Actor’s Future (GH#2374) Matthew Rocklin * Support missing packages keyword in Client.get_versions (GH#2379) Matthew Rocklin * Fixup serializing masked arrays (GH#2373) Jim Crist ------------------------------------------------------------------- Thu Nov 22 22:46:56 UTC 2018 - Arun Persaud - update to version 1.24.2: * Add support for Bokeh 1.0 (GH#2348) (GH#2356) Matthew Rocklin * Fix regression that dropped support for Tornado 4 (GH#2353) Roy Wedge * Avoid deprecation warnings (GH#2355) (GH#2357) Matthew Rocklin * Fix typo in worker documentation (GH#2349) Tom Rochette ------------------------------------------------------------------- Mon Nov 12 05:55:43 UTC 2018 - Arun Persaud - update to version 1.24.1: * Use tornado’s builtin AnyThreadLoopEventPolicy (GH#2326) Matthew Rocklin * Adjust TLS tests for openssl 1.1 (GH#2331) Marius van Niekerk * Avoid setting event loop policy if within Jupyter notebook server (GH#2343) Matthew Rocklin * Add preload script to conf (GH#2325) Guillaume Eynard-Bontemps * Add serializer for Numpy masked arrays (GH#2335) Peter Killick * Use psutil.Process.oneshot (GH#2339) NotSqrt * Use worker SSL context when getting client from worker. (GH#2301) Anonymous ------------------------------------------------------------------- Tue Oct 30 03:06:05 UTC 2018 - Arun Persaud - update to version 1.24.0: * Remove Joblib Dask Backend from codebase (GH#2298) Matthew Rocklin * Include worker tls protocol in Scheduler.restart (GH#2295) Matthew Rocklin * Adapt to new Bokeh selection for 1.0 (GH#2292) Matthew Rocklin * Add explicit retry method to Future and Client (GH#2299) Matthew Rocklin * Point to main worker page in bokeh links (GH#2300) Matthew Rocklin * Limit concurrency when gathering many times (GH#2303) Matthew Rocklin * Add tls_cluster pytest fixture (GH#2302) Matthew Rocklin * Convert ConnectionPool.open and active to properties (GH#2304) Matthew Rocklin * change export_tb to format_tb (GH#2306) Eric Ma * Redirect joblib page to dask-ml (GH#2307) Matthew Rocklin * Include unserializable object in error message (GH#2310) Matthew Rocklin * Import Mapping, Iterator, Set from collections.abc in Python 3 (GH#2315) Gaurav Sheni * Extend Client.scatter docstring (GH#2320) Eric Ma * Update for new flake8 (GH#2321) Matthew Rocklin ------------------------------------------------------------------- Mon Oct 8 15:02:15 UTC 2018 - Arun Persaud - update to version 1.23.3: * Err in dask serialization if not a NotImplementedError (GH#2251) Matthew Rocklin * Protect against key missing from priority in GraphLayout (GH#2259) Matthew Rocklin * Do not pull data twice in Client.gather (GH#2263) Adam Klein * Add pytest fixture for cluster tests (GH#2262) Matthew Rocklin * Cleanup bokeh callbacks (GH#2261) (GH#2278) Matthew Rocklin * Fix bokeh error for memory_limit=None (GH#2255) Brett Naul * Place large keywords into task graph in Client.map (GH#2281) Matthew Rocklin * Remove redundant blosc threading code from protocol.numpy (GH#2284) Mike Gevaert * Add ncores to workertable (GH#2289) Matthew Rocklin * Support upload_file on files with no extension (GH#2290) Matthew Rocklin ------------------------------------------------------------------- Mon Sep 17 14:55:33 UTC 2018 - Arun Persaud - update to version 1.23.2: * Discard dependent rather than remove (#2250) Matthew Rocklin * Use dask_sphinx_theme Matthew Rocklin * Drop the Bokeh index page (#2241) John Kirkham * Revert change to keep link relative (#2242) Matthew Rocklin * docs: Fix broken AWS link in setup.rst file (#2240) Vladyslav Moisieienkov * Return cancelled futures in as_completed (#2233) Chris White ------------------------------------------------------------------- Sat Sep 8 04:35:20 UTC 2018 - Arun Persaud - update to version 1.23.1: * Raise informative error when mixing futures between clients (#2227) Matthew Rocklin * add byte_keys to unpack_remotedata call (#2232) Matthew Rocklin * Add documentation for gist/rawgit for get_task_stream (#2236) Matthew Rocklin * Quiet Client.close by waiting for scheduler stop signal (#2237) Matthew Rocklin * Display system graphs nicely on different screen sizes (#2239) Derek Ludwig * Mutate passed in workers dict in TaskStreamPlugin.rectangles (#2238) Matthew Rocklin ------------------------------------------------------------------- Sun Sep 2 17:03:46 UTC 2018 - arun@gmx.de - specfile: * remove devel from noarch * be more specific in %files section - update to version 1.23.0: * Add direct_to_workers to Client Matthew Rocklin * Add Scheduler.proxy to workers Matthew Rocklin * Implement Actors Matthew Rocklin * Fix tooltip (#2168) Loïc Estève * Fix scale / avoid returning coroutines (#2171) Joe Hamman * Clarify dask-worker –nprocs (#2173) Yu Feng * Concatenate all bytes of small messages in TCP comms (#2172) Matthew Rocklin * Add dashboard_link property (#2176) Jacob Tomlinson * Always offload to_frames (#2170) Matthew Rocklin * Warn if desired port is already in use (#2191) (#2199) Matthew Rocklin * Add profile page for event loop thread (#2144) Matthew Rocklin * Use dispatch for dask serialization, also add sklearn, pytorch (#2175) Matthew Rocklin * Handle corner cases with busy signal (#2182) Matthew Rocklin * Check self.dependencies when looking at tasks in memory (#2196) Matthew Rocklin * Add ability to log additional custom metrics from each worker (#2169) Loïc Estève * Fix formatting when port is a tuple (#2204) Loïc Estève * Describe what ZeroMQ is (#2211) Mike DePalatis * Tiny typo fix (#2214) Anderson Banihirwe * Add Python 3.7 to travis.yml (#2203) Matthew Rocklin * Add plot= keyword to get_task_stream (#2198) Matthew Rocklin * Add support for optional versions in Client.get_versions (#2216) Matthew Rocklin * Add routes for solo bokeh figures in dashboard (#2185) Matthew Rocklin * Be resilient to missing dep after busy signal (#2217) Matthew Rocklin * Use CSS Grid to layout status page on the dashboard (#2213) Derek Ludwig and Luke Canavan * Fix deserialization of queues on main ioloop thread (#2221) Matthew Rocklin * Add a worker initialization function (#2201) Guillaume EB * Collapse navbar in dashboard (#2223) Luke Canavan - changes from version 1.22.1: * Add worker_class= keyword to Nanny to support different worker types (GH#2147) Martin Durant * Cleanup intermittent worker failures (GH#2152) (GH#2146) Matthew Rocklin * Fix msgpack PendingDeprecationWarning for encoding=’utf-8’ (GH#2153) Olivier Grisel * Make bokeh coloring deterministic using hash function (GH#2143) Matthew Rocklin * Allow client to query the task stream plot (GH#2122) Matthew Rocklin * Use PID and counter in thread names (GH#2084) (GH#2128) Dror Birkman * Test that worker restrictions are cleared after cancellation (GH#2107) Matthew Rocklin * Expand resources in graph_to_futures (GH#2131) Matthew Rocklin * Add custom serialization support for pyarrow (GH#2115) Dave Hirschfeld * Update dask-scheduler cli help text for preload (GH#2120) Matt Nicolls * Added another nested parallelism test (GH#1710) Tom Augspurger * insert newline by default after TextProgressBar (GH#1976) Phil Tooley * Retire workers from scale (GH#2104) Matthew Rocklin * Allow worker to refuse data requests with busy signal (GH#2092) Matthew Rocklin * Don’t forget released keys (GH#2098) Matthew Rocklin * Update example for stopping a worker (GH#2088) `John A Kirkham`_ * removed hardcoded value of memory terminate fraction from a log message (GH#2096) Bartosz Marcinkowski * Adjust worker doc after change in config file location and treatment (GH#2094) Aurélien Ponte * Prefer gathering data from same host (GH#2090) Matthew Rocklin * Handle exceptions on deserialized comm with text error (GH#2093) Matthew Rocklin * Fix typo in docstring (GH#2087) Loïc Estève * Provide communication context to serialization functions (GH#2054) Matthew Rocklin * Allow name to be explicitly passed in publish_dataset (GH#1995) Marius van Niekerk * Avoid accessing Worker.scheduler_delay around yield point (GH#2074) Matthew Rocklin * Support TB and PB in format bytes (GH#2072) Matthew Rocklin * Add test for as_completed for loops in Python 2 (GH#2071) Matthew Rocklin * Allow adaptive to exist without a cluster (GH#2064) Matthew Rocklin * Have worker data transfer wait until recipient acknowledges (GH#2052) Matthew Rocklin * Support async def functions in Client.sync (GH#2070) Matthew Rocklin * Add asynchronous parameter to docstring of LocalCluster Matthew Rocklin * Normalize address before comparison (GH#2066) Tom Augspurger * Use ConnectionPool for Worker.scheduler Matthew Rocklin * Avoid reference cycle in str_graph Matthew Rocklin * Pull data outside of while loop in gather (GH#2059) Matthew Rocklin ------------------------------------------------------------------- Mon Jul 2 09:52:43 UTC 2018 - tchvatal@suse.com - Fix the PyYAML dependency ------------------------------------------------------------------- Sun Jul 1 02:12:07 UTC 2018 - arun@gmx.de - specfile: * updated dask requirement, added pyyaml ------------------------------------------------------------------- Sun Jun 24 15:47:45 UTC 2018 - arun@gmx.de - update to version 1.22.0: * Overhaul configuration (GH#1948) Matthew Rocklin * Replace get= keyword with scheduler= (GH#1959) Matthew Rocklin * Use tuples in msgpack (GH#2000) Matthew Rocklin and Marius van Niekerk * Unify handling of high-volume connections (GH#1970) Matthew Rocklin * Automatically scatter large arguments in joblib connector (GH#2020) (GH#2030) Olivier Grisel * Turn click Python 3 locales failure into a warning (GH#2001) Matthew Rocklin * Rely on dask implementation of sizeof (GH#2042) Matthew Rocklin * Replace deprecated workers.iloc with workers.values() (GH#2013) Grant Jenks * Introduce serialization families (GH#1912) Matthew Rocklin * Add PubSub (GH#1999) Matthew Rocklin * Add Dask stylesheet to documentation Matthew Rocklin * Avoid recomputation on partially-complete results (GH#1840) Matthew Rocklin * Use sys.prefix in popen for testing (GH#1954) Matthew Rocklin * Include yaml files in manifest Matthew Rocklin * Use self.sync so Client.processing works in asynchronous context (GH#1962) Henry Doupe * Fix bug with bad repr on closed client (GH#1965) Matthew Rocklin * Parse –death-timeout keyword in dask-worker (GH#1967) Matthew Rocklin * Support serializers in BatchedSend (GH#1964) Matthew Rocklin * Use normal serialization mechanisms to serialize published datasets (GH#1972) Matthew Rocklin * Add security support to LocalCluster. (GH#1855) Marius van Niekerk * add ConnectionPool.remove method (GH#1977) Tony Lorenzo * Cleanly close workers when scheduler closes (GH#1981) Matthew Rocklin * Add .pyz support in upload_file (GH#1781) @bmaisson * add comm to packages (GH#1980) Matthew Rocklin * Replace dask.set_options with dask.config.set Matthew Rocklin * Exclude versions of sortedcontainers which do not have .iloc. (GH#1993) Russ Bubley * Exclude gc statistics under PyPy (GH#1997) Marius van Niekerk * Manage recent config and dataframe changes in dask (GH#2009) Matthew Rocklin * Cleanup lingering clients in tests (GH#2012) Matthew Rocklin * Use timeouts during Client._ensure_connected (GH#2011) Martin Durant * Avoid reference cycle in joblib backend (GH#2014) Matthew Rocklin, also Olivier Grisel * DOC: fixed test example (GH#2017) Tom Augspurger * Add worker_key parameter to Adaptive (GH#1992) Matthew Rocklin * Prioritize tasks with their true keys, before stringifying (GH#2006) Matthew Rocklin * Serialize worker exceptions through normal channels (GH#2016) Matthew Rocklin * Include exception in progress bar (GH#2028) Matthew Rocklin * Avoid logging orphaned futures in All (GH#2008) Matthew Rocklin * Don’t use spill-to-disk dictionary if we’re not spilling to disk Matthew Rocklin * Only avoid recomputation if key exists (GH#2036) Matthew Rocklin * Use client connection and serialization arguments in progress (GH#2035) Matthew Rocklin * Rejoin worker client on closing context manager (GH#2041) Matthew Rocklin * Avoid forgetting erred tasks when losing dependencies (GH#2047) Matthew Rocklin * Avoid collisions in graph_layout (GH#2050) Matthew Rocklin * Avoid recursively calling bokeh callback in profile plot (GH#2048) Matthew Rocklin ------------------------------------------------------------------- Sun May 6 05:30:39 UTC 2018 - arun@gmx.de - specfile: * Replace msgpack-python by msgpack * updated required versions according to requirement.txt - update to version 1.21.8: * Remove errant print statement (GH#1957) Matthew Rocklin * Only add reevaluate_occupancy callback once (GH#1953) Tony Lorenzo - changes from version 1.21.7: * Newline needed for doctest rendering (GH#1917) Loïc Estève * Support Client._repr_html_ when in async mode (GH#1909) Matthew Rocklin * Add parameters to dask-ssh command (GH#1910) Irene Rodriguez * Santize get_dataset trace (GH#1888) John Kirkham * Fix bug where queues would not clean up cleanly (GH#1922) Matthew Rocklin * Delete cached file safely in upload file (GH#1921) Matthew Rocklin * Accept KeyError when closing tornado IOLoop in tests (GH#1937) Matthew Rocklin * Quiet the client and scheduler when gather(…, errors=’skip’) (:pr:`1936) Matthew Rocklin * Clarify couldn’t gather keys warning (GH#1942) Kenneth Koski * Support submit keywords in joblib (GH#1947) Matthew Rocklin * Avoid use of external resources in bokeh server (GH#1934) Matthew Rocklin * Drop __contains__ from Datasets (GH#1889) John Kirkham * Fix bug with queue timeouts (GH#1950) Matthew Rocklin * Replace msgpack-python by msgpack (GH#1927) Loïc Estève ------------------------------------------------------------------- Sat Apr 7 19:25:08 UTC 2018 - arun@gmx.de - update to version 1.21.6: * Fix numeric environment variable configuration (GH#1885) Joseph Atkins-Kurkish * support bytearrays in older lz4 library (GH#1886) Matthew Rocklin * Remove started timeout in nanny (GH#1852) Matthew Rocklin * Don’t log errors in sync (GH#1894) Matthew Rocklin * downgrade stale lock warning to info logging level (GH#1890) Matthew Rocklin * Fix UnboundLocalError for key (GH#1900) John Kirkham * Resolve deployment issues in Python 2 (GH#1905) Matthew Rocklin * Support retries and priority in Client.get method (GH#1902) Matthew Rocklin * Add additional attributes to task page if applicable (GH#1901) Matthew Rocklin * Add count method to as_completed (GH#1897) Matthew Rocklin * Extend default timeout to 10s (GH#1904) Matthew Rocklin - changes from version 1.21.5: * Increase default allowable tick time to 3s (GH#1854) Matthew Rocklin * Handle errant workers when another worker has data (GH#1853) Matthew Rocklin * Close multiprocessing queue in Nanny to reduce open file descriptors (GH#1862) Matthew Rocklin * Extend nanny started timeout to 30s, make configurable (GH#1865) Matthew Rocklin * Comment out the default config file (GH#1871) Matthew Rocklin * Update to fix bokeh 0.12.15 update errors (GH#1872) Matthew Rocklin * Downgrade Event Loop unresponsive warning to INFO level (GH#1870) Matthew Rocklin * Add fifo timeout to control priority generation (GH#1828) Matthew Rocklin * Add retire_workers API to Client (GH#1876) Matthew Rocklin * Catch NoSuchProcess error in Nanny.memory_monitor (GH#1877) Matthew Rocklin * Add uid to nanny queue communitcations (GH#1880) Matthew Rocklin - changes from version 1.21.4: * Avoid passing bytearrays to snappy decompression (GH#1831) Matthew Rocklin * Specify IOLoop in Adaptive (GH#1841) Matthew Rocklin * Use connect-timeout config value throughout client (GH#1839) Matthew Rocklin * Support direct= keyword argument in Client.get (GH#1845) Matthew Rocklin - changes from version 1.21.3: * Add cluster superclass and improve adaptivity (GH#1813) Matthew Rocklin * Fixup tests and support Python 2 for Tornado 5.0 (GH#1818) Matthew Rocklin * Fix bug in recreate_error when dependencies are dropped (GH#1815) Matthew Rocklin * Add worker time to live in Scheduler (GH#1811) Matthew Rocklin * Scale adaptive based on total_occupancy (GH#1807) Matthew Rocklin * Support calling compute within worker_client (GH#1814) Matthew Rocklin * Add percentage to profile plot (GH#1817) Brett Naul * Overwrite option for remote python in dask-ssh (GH#1812) Sven Kreiss - changes from version 1.21.2: * Fix bug where we didn’t check idle/saturated when stealing (GH#1801) Matthew Rocklin * Fix bug where client was noisy when scheduler closed unexpectedly (GH#1806) Matthew Rocklin * Use string-based timedeltas (like '500 ms') everywhere (GH#1804) Matthew Rocklin * Keep logs in scheduler and worker even if silenced (GH#1803) Matthew Rocklin * Support minimum, maximum, wait_count keywords in Adaptive (GH#1797) Jacob Tomlinson and Matthew Rocklin * Support async protocols for LocalCluster, replace start= with asynchronous= (GH#1798) Matthew Rocklin * Avoid restarting workers when nanny waits on scheduler (GH#1793) Matthew Rocklin * Use IOStream.read_into() when available (GH#1477) Antoine Pitrou * Reduce LocalCluster logging threshold from CRITICAL to WARN (GH#1785) Andy Jones * Add futures_of to API docs (GH#1783) John Kirkham * Make diagnostics link in client configurable (GH#1810) Matthew Rocklin - changes from version 1.21.1: * Fixed an uncaught exception in distributed.joblib with a LocalCluster using only threads (GH#1775) Tom Augspurger * Format bytes in info worker page (GH#1752) Matthew Rocklin * Add pass-through arguments for scheduler/worker –preload modules. (GH#1634) Alexander Ford * Use new LZ4 API (GH#1757) Thrasibule * Replace dask.optimize with dask.optimization (GH#1754) Matthew Rocklin * Add graph layout engine and bokeh plot (GH#1756) Matthew Rocklin * Only expand name with –nprocs if name exists (GH#1776) Matthew Rocklin * specify IOLoop for stealing PeriodicCallback (GH#1777) Matthew Rocklin * Fixed distributed.joblib with no processes Tom Augspurger * Use set.discard to avoid KeyErrors in stealing (GH#1766) Matthew Rocklin * Avoid KeyError when task has been released during steal (GH#1765) Matthew Rocklin * Add versions routes to avoid the use of run in Client.get_versions (GH#1773) Matthew Rocklin * Add write_scheduler_file to Client (GH#1778) Joe Hamman * Default host to tls:// if tls information provided (GH#1780) Matthew Rocklin ------------------------------------------------------------------- Tue Mar 20 21:00:30 UTC 2018 - jengelh@inai.de - Update descriptions. ------------------------------------------------------------------- Sun Mar 11 08:51:03 UTC 2018 - sebix+novell.com@sebix.at - Use %license macro for license. ------------------------------------------------------------------- Sat Feb 10 17:28:26 UTC 2018 - arun@gmx.de - specfile: * update copyright year - update to version 1.21.0: * Refactor scheduler to use TaskState objects rather than dictionaries (:pr:`1594`) Antoine Pitrou * Plot CPU fraction of total in workers page (:pr:`1624`) Matthew Rocklin * Use thread CPU time in Throttled GC (:pr:`1625`) Antoine Pitrou * Fix bug with memory_limit=None (:pr:`1639`) Matthew Rocklin * Add futures_of to top level api (:pr:`1646`) Matthew Rocklin * Warn on serializing large data in Client (:pr:`1636`) Matthew Rocklin * Fix intermittent windows failure when removing lock file (:pr:`1652`) Antoine Pitrou * Add diagnosis and logging of poor GC Behavior (:pr:`1635`) Antoine Pitrou * Add client-scheduler heartbeats (:pr:`1657`) Matthew Rocklin * Return dictionary of worker info in retire_workers (:pr:`1659`) Matthew Rocklin * Ensure dumps_function works with unhashable functions (:pr:`1662`) Matthew Rocklin * Collect client name ids rom client-name config variable (:pr:`1664`) Matthew Rocklin * Allow simultaneous use of --name and --nprocs in dask-worker (:pr:`1665`) Matthew Rocklin * Add support for grouped adaptive scaling and adaptive behavior overrides (:pr:`1632`) Alexander Ford * Share scheduler RPC between worker and client (:pr:`1673`) Matthew Rocklin * Allow retries= in ClientExecutor (:pr:`1672`) @rqx * Improve documentation for get_client and dask.compute examples (:pr:`1638`) Scott Sievert * Support DASK_SCHEDULER_ADDRESS environment variable in worker (:pr:`1680`) Matthew Rocklin * Support tuple-keys in retries (:pr:`1681`) Matthew Rocklin * Use relative links in bokeh dashboard (:pr:`1682`) Matthew Rocklin * Make message log length configurable, default to zero (:pr:`1691`) Matthew Rocklin * Deprecate Client.shutdown (:pr:`1699`) Matthew Rocklin * Add warning in configuration docs to install pyyaml (:pr:`1701`) Cornelius Riemenschneider * Handle nested parallelism in distributed.joblib (:pr:`1705`) `Tom Augspurger`_ * Don't wait for Worker.executor to shutdown cleanly when restarting process (:pr:`1708`) Matthew Rocklin * Add support for user defined priorities (:pr:`1651`) Matthew Rocklin * Catch and log OSErrors around worker lock files (:pr:`1714`) Matthew Rocklin * Remove worker prioritization. Coincides with changes to dask.order (:pr:`1730`) Matthew Rocklin * Use process-measured memory rather than nbytes in Bokeh dashboard (:pr:`1737`) Matthew Rocklin * Enable serialization of Locks (:pr:`1738`) Matthew Rocklin * Support Tornado 5 beta (:pr:`1735`) Matthew Rocklin * Cleanup remote_magic client cache after tests (:pr:`1743`) Min RK * Allow service ports to be specified as (host, port) (:pr:`1744`) Bruce Merry ------------------------------------------------------------------- Thu Jan 11 19:40:54 UTC 2018 - toddrme2178@gmail.com - Fix futures dependency handling ------------------------------------------------------------------- Fri Dec 8 18:05:45 UTC 2017 - arun@gmx.de - update to version 1.20.2: * Clear deque handlers after each test (GH#1586) Antoine Pitrou * Handle deserialization in FutureState.set_error (GH#1592) Matthew Rocklin * Add process leak checker to tests (GH#1596) Antoine Pitrou * Customize process title for subprocess (GH#1590) Antoine Pitrou * Make linting a separate CI job (GH#1599) Antoine Pitrou * Fix error from get_client() with no global client (GH#1595) Daniel Li * Remove Worker.host_health, correct WorkerTable metrics (GH#1600) Matthew Rocklin * Don’t mark tasks as suspicious when retire_workers called. Addresses (GH#1607) `Russ Bubley`_ * Do not include processing workers in workers_to_close (GH#1609) `Russ Bubley`_ * Disallow simultaneous scale up and down in Adaptive (GH#1608) `Russ Bubley`_ * Parse bytestrings in –memory-limit (GH#1615) Matthew Rocklin * Use environment variable for scheduler address if present (GH#1610) Matthew Rocklin * Fix deprecation warning from logger.warn (GH#1616) Brett Naul ------------------------------------------------------------------- Mon Nov 27 19:31:41 UTC 2017 - arun@gmx.de - update to version 1.20.1: * Wrap import ssl statements with try-except block for ssl-crippled environments, (GH#1570) Xander Johnson * Support zero memory-limit in Nanny (GH#1571) Matthew Rocklin * Avoid PeriodicCallback double starts (GH#1573) Matthew Rocklin * Add disposable workspace facility (GH#1543) Antoine Pitrou * Use format_time in task_stream plots (GH#1575) Matthew Rocklin * Avoid delayed finalize calls in compute (GH#1577) Matthew Rocklin * Doc fix about secede (GH#1583) Scott Sievert * Add tracemalloc option when tracking test leaks (GH#1585) Antoine Pitrou * Add JSON routes to Bokeh server (GH#1584) Matthew Rocklin * Handle exceptions cleanly in Variables and Queues (GH#1580) Matthew Rocklin ------------------------------------------------------------------- Sun Nov 19 05:13:33 UTC 2017 - arun@gmx.de - specfile: * only require python-futures for python2 * removed files from install section that are not in the tar-ball anymore - update to version 1.20.0: * Drop use of pandas.msgpack (GH#1473) Matthew Rocklin * Add methods to get/set scheduler metadata Matthew Rocklin * Add distributed lock Matthew Rocklin * Add reschedule exception for worker tasks Matthew Rocklin * Fix nbytes() for bytearrays Matthew Rocklin * Capture scheduler and worker logs Matthew Rocklin * Garbage collect after data eviction on high worker memory usage (GH#1488) Olivier Grisel * Add scheduler HTML routes to bokeh server (GH#1478) (GH#1514) Matthew Rocklin * Add pytest plugin to test for resource leaks (GH#1499) Antoine Pitrou * Improve documentation for scheduler states (GH#1498) Antoine Pitrou * Correct warn_if_longer timeout in ThrottledGC (GH#1496) Fabian Keller * Catch race condition in as_completed on cancelled futures (GH#1507) Matthew Rocklin * Transactional work stealing (GH#1489) (GH#1528) Matthew Rocklin * Avoid forkserver in PyPy (GH#1509) Matthew Rocklin * Add dict access to get/set datasets (GH#1508) Mike DePalatis * Support Tornado 5 (GH#1509) (GH#1512) (GH#1518) (GH#1534) Antoine Pitrou * Move thread_state in Dask (GH#1523) Jim Crist * Use new Dask collections interface (GH#1513) Matthew Rocklin * Add nanny flag to dask-mpi Matthew Rocklin * Remove JSON-based HTTP servers Matthew Rocklin * Avoid doing I/O in repr/str (GH#1536) Matthew Rocklin * Fix URL for MPI4Py project (GH#1546) Ian Hopkinson * Allow automatic retries of a failed task (GH#1524) Antoine Pitrou * Clean and accelerate tests (GH#1548) (GH#1549) (GH#1552) (GH#1553) (GH#1560) (GH#1564) Antoine Pitrou * Move HDFS functionality to the hdfs3 library (GH#1561) Jim Crist * Fix bug when using events page with no events (GH#1562) @rbubley * Improve diagnostic naming of tasks within tuples (GH#1566) Kelvyn Yang ------------------------------------------------------------------- Thu Nov 9 18:54:37 UTC 2017 - arun@gmx.de - specfile: * added binary dask-mpi - update to version 1.19.3: * Handle None case in profile.identity (GH#1456) * Asyncio rewrite (GH#1458) * Add rejoin function partner to secede (GH#1462) * Nested compute (GH#1465) * Use LooseVersion when comparing Bokeh versions (GH#1470) - changes from version 1.19.2: * as_completed doesn’t block on cancelled futures (GH#1436) * Notify waiting threads/coroutines on cancellation (GH#1438) * Set Future(inform=True) as default (GH#1437) * Rename Scheduler.transition_story to story (GH#1445) * Future uses default client by default (GH#1449) * Add keys= keyword to Client.call_stack (GH#1446) * Add get_current_task to worker (GH#1444) * Ensure that Client remains asynchornous before ioloop starts (GH#1452) * Remove “click for worker page” in bokeh plot (GH#1453) * Add Client.current() (GH#1450) * Clean handling of restart timeouts (GH#1442) - changes from version 1.19.1: * Fix tool issues with TaskStream plot (GH#1425) * Move profile module to top level (GH#1423) - changes from version 1.19.0: * Avoid storing messages in message log (GH#1361) * fileConfig does not disable existing loggers (GH#1380) * Offload upload_file disk I/O to separate thread (GH#1383) * Add missing SSLContext (GH#1385) * Collect worker thread information from sys._curent_frames (GH#1387) * Add nanny timeout (GH#1395) * Restart worker if memory use goes above 95% (GH#1397) * Track workers memory use with psutil (GH#1398) * Track scheduler delay times in workers (GH#1400) * Add time slider to profile plot (GH#1403) * Change memory-limit keyword to refer to maximum number of bytes (GH#1405) * Add cancel(force=) keyword (GH#1408) - changes from version 1.18.2: * Silently pass on cancelled futures in as_completed (GH#1366) * Fix unicode keys error in Python 2 (GH#1370) * Support numeric worker names * Add dask-mpi executable (GH#1367) - changes from version 1.18.1: * Clean up forgotten keys in fire-and-forget workloads (GH#1250) * Handle missing extensions (GH#1263) * Allow recreate_exception on persisted collections (GH#1253) * Add asynchronous= keyword to blocking client methods (GH#1272) * Restrict to horizontal panning in bokeh plots (GH#1274) * Rename client.shutdown to client.close (GH#1275) * Avoid blocking on event loop (GH#1270) * Avoid cloudpickle errors for Client.get_versions (GH#1279) * Yield on Tornado IOStream.write futures (GH#1289) * Assume async behavior if inside a sync statement (GH#1284) * Avoid error messages on closing (GH#1297), (GH#1296) (GH#1318) (GH#1319) * Add timeout= keyword to get_client (GH#1290) * Respect timeouts when restarting (GH#1304) * Clean file descriptor and memory leaks in tests (GH#1317) * Deprecate Executor (GH#1302) * Add timeout to ThreadPoolExecutor.shutdown (GH#1330) * Clean up AsyncProcess handling (GH#1324) * Allow unicode keys in Python 2 scheduler (GH#1328) * Avoid leaking stolen data (GH#1326) * Improve error handling on failed nanny starts (GH#1337), (GH#1331) * Make Adaptive more flexible * Support --contact-address and --listen-address in worker (GH#1278) * Remove old dworker, dscheduler executables (GH#1355) * Exit workers if nanny process fails (GH#1345) * Auto pep8 and flake (GH#1353) ------------------------------------------------------------------- Mon May 22 19:14:46 UTC 2017 - toddrme2178@gmail.com - Initial version