------------------------------------------------------------------- Mon Apr 4 09:08:29 UTC 2022 - John Paul Adrian Glaubitz - Update to 2022.3.0 Enhancements * tqdm example callback with simple methods (#931, 902) * Allow empty root in get_mapper (#930) * implement real info for reference FS (#919) * list known implementations and compressions (#913) Fixes * git branch for testing git backend (#929) * maintaine mem FS's root (#926) * kargs to FS in parquet module (#921) * fix on_error in references (#917) * tar ls consistency (#9114) * pyarrow: don't decompress twice (#911) * fix FUSE tests (#905) ------------------------------------------------------------------- Tue Feb 22 23:01:18 UTC 2022 - Matej Cepl - Update to 2022.02.0: - reference FS performance - directory/prefix FS - FUSE - iteration in threads - OpenFiles slicing - drop py36 ------------------------------------------------------------------- Mon Jan 24 17:08:51 UTC 2022 - Ben Greiner - Update to 2022.1.0 * Fix blocks cache metadata (#746) * Fix default SMB port (#853) * Fix caching fixes (#856, 855) * Fix explicit close for http files (#866) * Fix put_file to continue when no bytes (#869, 870) * temporary files location (#851, 871) * async abstract methods (#858, 859, 860) * md5 for FIPS (#872) * remove deprecated pyarrow/distutils (#880, 881) - Update flavored requirements: python36 no longer exists, python-distributed does not support python310 yet. ------------------------------------------------------------------- Fri Dec 24 23:06:53 UTC 2021 - Ben Greiner - Release 2021.11.1 Enhancements * allow compression for fs.open (#826) * batch more async operations (#824) * allow github backend for alternate URL (#815) * speec up reference filesystem (#811) Fixes * fixes for parquet functionality (#821, 817) * typos and docs (#839, 833, 816) * local root (#829) Other * remove BlockSizeError for http (#830) - Release 2021.11.0 Enhancement * parquet-specific module and cache type (#813, #806) Fixes * empty ranges (#802, 801, 803) * doc typos (#791, 808) * entrypoints processing (#784) * cat in ZIP (#789) Other * move to fsspec org * doc deps (#786, 791) - Release 2021.10.1 Fixes * Removed inaccurate ZipFileSystem.cat() override so that the base class' version is used (#789) * fix entrypoint processing (#784) * case where no blocks of a block-cache have yet been loaded (#801) * don't fetch empty ranges (#802, 803) Other * simplify doc deps (#786, 791) - Release 2021.10.0 Fixes * only close http connector if present (#779) * hdfs strip protocol (#778) * fix filecache with check_files (#772) * put_file to use _parent (#771) Other * add kedro link (#781) - Release 2021.09.0 Enhancement * http put from file-like (#764) * implement webhdfs cp/rm_file (#762) * multiple (and concurrent) cat_ranges (#744) Fixes * sphinx warnings (#769) * lexists for links (#757) * update versioneer (#750) * hdfs updates (#749) * propagate async timeout error (#746) * fix local file seekable (#743) * fix http isdir when does not exist (#741) Other * ocifs, arrow added (#754, #765) * promote url_to_fs to top level (#753) ------------------------------------------------------------------- Tue Sep 21 10:00:47 UTC 2021 - Ben Greiner - Update to 2021.8.1 * HTTP get_file/put_file APIs now support callbacks (#731) * New HTTP put_file method for transferring data to the remote server (chunked) (#731) * Customizable HTTP client initializers (through passing get_client argument) (#731, #701) * Support for various checksum / fingerprint headers in HTTP info() (#731) * local implementation of rm_file (#736) * local speed improvements (#711) * sharing options in SMB (#706) * streaming cat/get for ftp (#700) * check for remote directory when putting (#737) * storage_option update handling (#734( * await HTTP call before checking status (#726) * ftp connect (#722) * bytes conversion of times in mapper (#721) * variable overwrite in WholeFileCache cat (#719) * http file size again (#718) * rm and create directories in ftp (#716, #703) * list of files in async put (#713) * bytes to dict in cat (#710) - Drop fsspec-pr710-bytesreturn.patch merged upstream ------------------------------------------------------------------- Mon Aug 23 17:04:01 UTC 2021 - Ben Greiner - Add fsspec-pr710-bytesreturn.patch in order to fix zarr * gh#intake/filesystem_spec#710 * gh#zarr-developers/zarr-python#812 ------------------------------------------------------------------- Sun Aug 8 15:32:54 UTC 2021 - Ben Greiner - Update to version 2021.07.0 * no release notes - Release 2021.06.1 * Introduce fsspec.asyn.fsspec_loop to temporarily switch to the fsspec loop. (#671) * support list for local rm (#678) * error when local mkdir twice (#679) * fix local info regression for pathlike (#667) * link to wandbfs (#664) - Release 2021.06.0 * Better testing and folder handling for Memory (#654) * Negative indexes for cat_file (#653) * optimize local file listing (#647) * FileNoteFound in http and range exception subclass (#649, 646) * async timeouts (#643, 645) * stringify path for pyarrow legacy (#630) * The fsspec.asyn.get_loop() will always return a loop of a selector policy (#658) * add helper to construct Range headers for cat_file (#655) ------------------------------------------------------------------- Thu May 20 13:22:56 UTC 2021 - Ben Greiner - Update to 2021.05.0 * Enable listings cache for HTTP filesystem (#560) * Fold ZipFileSystem and LibArchiveFileSystem into a generic implementation and add new TarFileSystem (#561) * Use throttling for the ``get``/``put`` methods of ``AsyncFileSystem``. - Release 2021.04.0 * Major changes: calendar versioning * better link and size finding for HTTP (#610, %99) * link following in Local (#608) * ReferenceFileSystem dev (#606, #604, #602) * Fix: drop metadata dep (#605) ------------------------------------------------------------------- Thu Apr 15 13:10:09 UTC 2021 - Ben Greiner - Update to version 0.9 * avoid nested sync calls by copying code (#581, #586, docs #593) * release again for py36 (#564, #575) * logging in mmap cacher, explicitly close files (#559) * make LocalFileOpener an IOBase (#589) * better reference file system (#568, #582, #584, #585) * first-chunk cache (#580) * sftp listdir (#571) * http logging and fetch all (#551, #558) * doc: entry points (#548) * get_mapper for caching filesystems (#559) * fix cross-device file move (#547) * store paths without trailing "/" for DBFS (#557) * errors that happen on _initiate_upload when closing the AbstractBufferedFile will now be propagated (#587) * infer_compressions with upper case suffix ($595) * file initialiser errors (#587) * CI fix (#563) * local file commit cross-device (#547) - Version 0.8.7 * fix error with pyarrow metadata for some pythons (#546) - Version 0.8.6 * Add dbfs:// support (#504, #514) * don't import pyarrow (#503) * update entry points syntax (#515) * ci precommit hooks (#534) * random appending of a directory within the filesystems find() method (#507, 537) * fix git tests (#501) * fix recursive memfs operations (#502) * fix recorsive/maxdepth for cp (#508) * fix listings cache timeout (#513) * big endian bytes tests (#519) * docs syntax (#535, 524, 520, 542) * transactions and reads (#533) - Don't test with distributed on python36 -- not available anymore ------------------------------------------------------------------- Mon Feb 22 16:55:37 UTC 2021 - Ben Greiner - Update python36 requirements. Don't try to test nonexistent optional python36-numpy and -panel ------------------------------------------------------------------- Mon Jan 4 17:59:57 UTC 2021 - Benjamin Greiner - Test in _multibuild in order to break build dependency cycles ------------------------------------------------------------------- Mon Dec 28 15:19:45 UTC 2020 - Benjamin Greiner - Update to 0.8.5 Features: * config system * libarchive implementation * add reference file system implementation - Changes in version 0.8.4 Features: * function can_be_local to see whether URL is compatible with open_local * concurrent cat with filecaches, if backend supports it * jupyter FS Fixes: * dircache expiry after transaction * blockcache garbage collection * close for HDFS * windows tests * glob depth with "**" - Changes in version 0.8.3 Features: * error options for cat * memory fs created time in detailed ls Fixes: * duplicate directories could appear in MemoryFileSystem * Added support for hat dollar lbrace rbrace regex character escapes in glob * Fix blockcache (was doing unnecessary work) * handle multibyte dtypes in readinto * Fix missing kwargs in call to _copy in asyn Other: * Stop inheriting from pyarrow.filesystem for pyarrow>=2.0 * Raise low-level program friendly OSError. * Guard against instance reuse in new processes * Make hash_name a method on CachingFileSystem to make it easier to change. * Use get_event_loop for py3.6 compatibility - Changes in version 0.8.2 Fixes: * More careful strip for caching * Version 0.8.1 Features: * add sign to base class * Allow calling of coroutines from normal code when running async * Implement writing for cached many files * Allow concurrent caching of remote files * Add gdrive:// protocol Fixes: * Fix memfs with exact ls * HTTPFileSystem requires requests and aiohttp in registry Other: * Allow http kwargs to clientSession * Use extras_require in setup.py for optional dependencies * Replacing md5 with sha256 for hash (CVE req) * Test against Python 3.8, drop 3.5 testing * add az alias for abfs - Get source tarball from PyPI instead of GitHub - BuildRequires and Suggests extra stuff and test it, where possible ------------------------------------------------------------------- Mon Sep 7 13:23:31 UTC 2020 - John Paul Adrian Glaubitz - Update to 0.8.0: * Change channel order for s3fs test * Add aiobotocore to dpes for s3fs check * test_local: fix username not in home path (#253) * Tighten cacheFS if dir deleted and open_local behaviour * Add test and docs * Attempt at context for multiple files * #355 typo fixed * Fix race condition of lzma import when using threads * Small change to methods in async not to overwrite * remove debugging print * properly rewind MemoryFile * Make sync functions in asyn to avoid nedsted cals to sync() * Bail when unchaining if only one element * Update fsspec/tests/test_api.py * Fix local mv * Document async, and ensure docstrings * included smb:// in api docs * fix smbprotocol[kerberos] docs on smb:// * smb:// wrap in SMBFileOpener instead of patching open_file * pass glob kwargs to super class from local fs * add test for local fs glob detail arg * improve docs for smb protocol * clean memfs * fix CI failure for remote black complaining extra commas * refactor smb functions from smv review issue * fixed many pylint nitpicks in smb suport modules * Remove chunk access; add mapper methods * fixed typo "gui" => "gid" * address review issues of smb:// protocol PR #340 * Use downstream URL paths when using {block,file,simple}cache in the middle. * add smbprotocol as deps for the CI build * handle smb:// protocol using smbprotocol package * Implement maybe_sync and text for using http in async * Add "pipe" and async version * Fix caching (needed cat_file) * Set default clobber=True on impl register * Add docs, more methods and one test-ish * Use skip_instance_cache instead of using a fixture for blockcache_workflow test * Show that chaining zip::zip::file works * Reproduce #334 in a test case * Convert unpickled cached file blocks info from list to set. * Async restructure and back-n-forth * Show #333 can be replicated * Create a disable_fs_caching pytest fixture * Revert "Force MMapCache blocks to "set" type" * Use "/".join instead of os.path.join in tests. * Try to replicate the blockcache bug * Use self._parent instead of os.dirname * Remove ZipFS feature flag to use AbstractFS info implementation * Test find behavior for ZipFileSystem * Use fsspec.filesystem("zip", ...) instead of get_filesystem_cls("zip")(...) * Force MMapCache blocks to "set" type * Remove unused monkeypatch, and use 1000 instead of 1_000 * Only use the base vs cache implementation when we compare the results directly * Add flag to select the old zip.info() implementation * Add pytest-benchmark as dev dependency * Add cached version of ZipFileSystem.info * Test implicit directories as well. * Add unit test for zip.info * Add s3fs dep (was there before?) * Use _get_kwargs_from_url when unchaining * Fix local file ops * Don't check exists in expand_path * Reduce cache miss on read-ahead * mismatch test & shutil.move * re-allow Path object * Add mcat to http * Add newline to openfile reduce * Test multi-download * fix memfs dir recursive and add test * Fix recursive and some memfs dirs * add aiohttp to test deps * Tighten up mkdir/rmdir for memfs * Fix cache for downloads * add doc example and test * Make simplecache writable * Add cache_type and cache_options to HTTPFileSystem constructore * Do unchain one level deeper * Make LocalFileOpener iterable * Fix pickling of LocalFileOpener ------------------------------------------------------------------- Wed Jun 24 03:58:37 UTC 2020 - Steve Kowalik - Update to 0.7.4: * No upstream changelog. - Add numpy to BuildRequires. ------------------------------------------------------------------- Wed Nov 20 14:14:49 UTC 2019 - Tomáš Chvátal - Update to 0.6.0: * Fixed issues with filesystem instance caching. This was causing authorization errors in downstream libraries like gcsfs and s3fs in multi-threaded code (:pr:`155`, :pr:`181`) * Changed the default file caching strategy to :class:`fsspec.caching.ReadAheadCache` (:pr:`193`) * Moved file caches to the new fsspec.caching module. They're still available from their old location in fsspec.core, but we recommend using the new location for new code (:pr:`195`) * Added a new file caching strategy, :class:`fsspec.caching.BlockCache` for fetching and caching file reads in blocks (:pr:`191`). * Fixed equality checks for file system instance to return False when compared to objects other than file systems (:pr:`192`) * Fixed a bug in :meth:`fsspec.FSMap.keys` returning a generator, which was consumed upon iteration (:pr:`189`). * Removed the magic addition of aliases in AbstractFileSystem.__init__. Now alias methods are always present (:pr:`177`) * Deprecated passing trim to :class:`fsspec.spec.AbstractBufferedFile`. Pass it in storage_options instead (:pr:`188`) * Improved handling of requests for :class:`fsspec.implementations.http.HTTPFileSystem` when the HTTP server responds with an (incorrect) content-length of 0 (:pr:`163`) * Added a detail=True parameter to :meth:`fsspec.spec.AbstractFileSystem.ls` (:pr:`168`) * Fixed handling of UNC/DFS paths (:issue:`154`) ------------------------------------------------------------------- Wed Oct 16 13:06:56 UTC 2019 - Tomáš Chvátal - Update to 0.5.2: * no upstream changelog ------------------------------------------------------------------- Tue Sep 24 11:48:55 UTC 2019 - Tomáš Chvátal - Update to 0.5.1: * no upstream changelog ------------------------------------------------------------------- Thu Sep 12 11:59:10 UTC 2019 - Tomáš Chvátal - Update to 0.4.4: * no upstream changelog ------------------------------------------------------------------- Thu Aug 1 09:17:56 UTC 2019 - Tomáš Chvátal - Update to 0.4.0: * No upstream changelog ------------------------------------------------------------------- Tue Jul 30 14:02:33 UTC 2019 - pgajdos@suse.com - version update to 0.3.6 * no upstream change log found ------------------------------------------------------------------- Mon Jul 22 16:32:08 UTC 2019 - Todd R - Initial version