* allow override of expand in open()
* root handling in local file paths, fix for windows
* coroutines throttle to stream pool rather than batches
* write transactions in simplecache
* allow deep nested refs in referenceFS/parquet
* Fixes bug (#1476) that made open_files ignore expand=False
* remove extra calling mapper contains
* connection retry for SMB
* zip64 should be on is allowZip64 is
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-fsspec?expand=0&rev=66
- Update to 2024.2.0
## Enhancements
* add 9P known implementation (#1513)
* allow TqdmCallback subclassing (#1497, 1480)
* callbacks/branching kwargs handling and scopes (#1496, 1495,
1460)
* add aluuxioFS to known implementations (#1469)
* implement pipe_file for dirFS (#1465)
## Fixes
* infer compression for .lzma files (#1514)
* fix append to categorical/parquet references (#1510)
* allow for FTP servers that list with leading "total" line
(#1503)
* convert FTP failure to FileNotFound (#1494)
* out of order reference fix (#1492)
* retry "backoff" response for DBFS (#1491)
* referenceFS case for scalar arrays (#1487)
* fix create_parents for SFTP (#1484)
* fix local .ls() on files (#1479)
* allow Path and similar in _expand_path (#1475)
* make lazy references editable (#1468)
* fix eq for abstract buffered files (#1466)
* fit tqdm cleanup (#1463)
* fix passing kwargs from cached file to underlying FS (#1462)
## Other
* fix tests for supports_empty_directories=False (#1512)
* don't read references in init for referenceFS (#1521)
* code cleaning (#1518, 1502, 1499, 1493, 1481)
* pass through "replication" for HDFS (#1486)
* record more info for HTTP info() (#1483)
OBS-URL: https://build.opensuse.org/request/show/1156422
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-fsspec?expand=0&rev=64
* Remove trailing "/" from directory names in zipFS (#1445)
* allow HTTP size guess in more circumstances (#1440)
* allow kwargs passed to GUI to be dict (#1437)
* transaction support for writing via a cache FS (#1434)
* make cached FSs work better with async backends (#1429)
* allow FSs to set their transaction implementation (#1424)
* add dataFS (#1421, 1415)
* allow basic auth in webHDFS (#1409)
* in referenceFS, maintain order when some keys are omitted in
cat (#1436)
* nested subdirectory listing in referenceFS (#1433)
* allow "=" in webHDF paths (#1428)
* fix file mode to consistent "r+b" format (#1426)
* pass on kwargs in HTTP glob (#1422)
* allow Path in can_be_local and open_local (#1419, #1418)
* fix parent for cachedFS (#1413)
* "ends" list in _cat_ranges was incorrect (#1402)
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-fsspec?expand=0&rev=58
* revert #1358: auto_mkdir in open() (#1365)
* code style updates (#1373, 1372, 1371, 1370, 1369, 1364)
* update CI setup (#1386)
- 2023.9.1
* #1353, save file cache metadata in JSON
* #1352, remove some unnecessary list iterations
* #1361, re-allow None for default port for SMB
* #1360, initialising GUI widget FS with kwargs
* #1358, pass auto_mkdir vi url_to_fs again
* #1354, auto delete temp cache directory
- 2023.9.0
* #1346, add ocilake protocol
* #1345, implement async-sync and async-async generic cp and
rsync
* #1344, add lakefs protocol
* #1337 add goatcounter to docs
* #1323, 1328, add xethub protocol
* #1320, in HTTP, check content-encoding when getting length
* #1303, add on_error in walk
* #1302, add dirfs attribute to mappers
* #1293, configure port for smb
* #1349, don't reorder paths in bulk ops if source and dest are
both lists
* #1333, allow mode="x" in get_fs_token_paths
* #1324, allow generic to work with complex URLs
* #1316, exclude bytes-cache kwargs in url_to_fs
* #1314, remote utcnow/utcfromtimestamp
* #1311, dirFS's protocol
* #1305, use get_file rather than get in file caching
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-fsspec?expand=0&rev=54
* #1236, allow writing ReferenceFS references directly to
parquet
* #1255, copy of glob to single output directory
* #1254, non-recursive copy of directory (no-op)
* #1253, cleanup fix on close of ZIP FS
* #1250, ignore dirs when copying list of files
* #1249, don't error on register without clobber is registering
same thing again
* #1245, special case for other_files and relative path
* Other
* #1248, add test harness into released wheel package
* #1247, docs and tests around common bulk file operations
* 2023.4.0
* #1225, comprehensive docs of expected behaviour of cp/get/put
and tests
* #1216, test harness for any backend
* #1224, small fixes in reference and dask FSs
* #1218, mv is no-op when origin and destination are the same
* #1217, await in AbstractStreamedFile
* #1215, docbuild fixes
* #1214, unneeded maxdepth manipulation in expand_path
* #1213, pyarros and posixpath related test fixes
* #1211, BackgroundBlockCache: keep a block longer if not yet
used
* #1210, webHDFS: location parameter
* Other
* #1241, add HfFileSystem to registry
* #1237, register_implementation clobber default changes to
False
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-fsspec?expand=0&rev=50
- Update to 2023.3.0
## Enhancements
* #1201, add directory FS to the registry and constructable from URLs
* #1194, allow JSON for setting dict-like kwargs in the config
* #1181, give arrow FS proper place in the registry
* #1178, add experimental background-thread buffering cache
* #1162, make ZipFS writable
## Fixes
* #1202, fix on_error="omit" when using caching's cat
* #1199, 1163, get/put/cp consistency and empty directories
* #1197, 1183 use bytes for setting value on mapper using numpy
* #1191, clean up open files in spec get_file
* #1164, pass on kwargs correctly to http
## Other
* #1186, make seekable=True default for pyarrow files
* #1184, 1185, set minimum python version to 3.8
- Release 2023.1.0
## Enhancements
* experimental DFReferenceFileSystem (#1157, 1138)
* pyarrow seeking (#1154)
* tar thread safety (#1132)
* fsid method (#1122)
## Fixes
* ReferenceFS order fix (#1158)
* fix du and maxdepth (#1128, 1151)
* http ranges (#1141)
## Other
* coverage on referenceFS (#1133, 1123)
* docs (#1152, 1150)
* remove code duplication in unchain (#1143, 1156, 1121)
OBS-URL: https://build.opensuse.org/request/show/1074232
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-fsspec?expand=0&rev=46
- Update to 2022.10.0
Enhancements
* referenceFS consolidates reads in the same remote file (#1063)
* localfs: add link/symlink/islink (#1059)
* asyncfs: make mirroring methods optional (#1054)
* local: info: provide st_ino and st_nlink from stat (#1053)
* arrow_hdfs replaces hdfs (#1051)
* Add read/write_text (#1047)
* Add pipe/cat to genericFS (#1038)
Fixes
* SSH write doesn’t return number of bytes (#1072)
* wrap flush method for LocalFileOpened (#1070)
* localfs: fix support for pathlib/os.PathLike objects in rm (#1058)
* don’t get_file remote FTP directory (#1056)
* fix zip write to remote (#1046)
* fix zip del following failed init (#1040)
Other
* add asynclocalfs to the registry (#1060)
* add DVCFileSystem to the registry (#1049)
* add downstream tests (#1037)
* Don’t auto-close OpenFiles (#1035)
OBS-URL: https://build.opensuse.org/request/show/1032000
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-fsspec?expand=0&rev=41
- Update to 2022.8.2
* don’t close OpenFile on del (#1035)
- Update to 2022.8.1
* revert #1024 (#1029), with strciter requirements on OpenFile usage
- Update to 2022.8.0
Enhancements
* writable ZipFileSystem (#1017)
* make OpenFile behave like files and remove dynamic closer in .open() (#1024)
* use isal gunzip (#1008)
Fixes
* remove strip from _parent (#1022)
* disallow aiohttp prereleases (#1018)
* be sure to close cached file (#1016)
* async rm in reverse order (#1014)
* expose fileno in LocalFileOpener (#1010, #1005)
* remove temp files with simplecache writing (#1006)
* azure paths (#1003)
copy dircache keys before iter
- Update to 2022.7.1
Fixes
* Remove fspath from LocalFileOpener (#1005)
* Revert 988 (#1003)
- Update to 2022.7.0
Enhancements
* added fsspec-xrootd implementation to registry (#1000)
* memory file not to copy bytes (#999)
* Filie details passed to FUSE (#972)
Fixes
* Return info for root path of archives (#996)
* arbitrary kwargs passed through in pipe_file (#993)
OBS-URL: https://build.opensuse.org/request/show/1006764
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-fsspec?expand=0&rev=37
- Update to 2021.8.1
* HTTP get_file/put_file APIs now support callbacks (#731)
* New HTTP put_file method for transferring data to the remote
server (chunked) (#731)
* Customizable HTTP client initializers (through passing
get_client argument) (#731, #701)
* Support for various checksum / fingerprint headers in HTTP
info() (#731)
* local implementation of rm_file (#736)
* local speed improvements (#711)
* sharing options in SMB (#706)
* streaming cat/get for ftp (#700)
* check for remote directory when putting (#737)
* storage_option update handling (#734(
* await HTTP call before checking status (#726)
* ftp connect (#722)
* bytes conversion of times in mapper (#721)
* variable overwrite in WholeFileCache cat (#719)
* http file size again (#718)
* rm and create directories in ftp (#716, #703)
* list of files in async put (#713)
* bytes to dict in cat (#710)
- Drop fsspec-pr710-bytesreturn.patch merged upstream
OBS-URL: https://build.opensuse.org/request/show/920841
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-fsspec?expand=0&rev=29
- Update to version 2021.07.0
* no release notes
- Release 2021.06.1
* Introduce fsspec.asyn.fsspec_loop to temporarily switch to the
fsspec loop. (#671)
* support list for local rm (#678)
* error when local mkdir twice (#679)
* fix local info regression for pathlike (#667)
* link to wandbfs (#664)
- Release 2021.06.0
* Better testing and folder handling for Memory (#654)
* Negative indexes for cat_file (#653)
* optimize local file listing (#647)
* FileNoteFound in http and range exception subclass (#649, 646)
* async timeouts (#643, 645)
* stringify path for pyarrow legacy (#630)
* The fsspec.asyn.get_loop() will always return a loop of a
selector policy (#658)
* add helper to construct Range headers for cat_file (#655)
OBS-URL: https://build.opensuse.org/request/show/910724
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-fsspec?expand=0&rev=27
- Update to version 0.9
* avoid nested sync calls by copying code (#581, #586, docs #593)
* release again for py36 (#564, #575)
* logging in mmap cacher, explicitly close files (#559)
* make LocalFileOpener an IOBase (#589)
* better reference file system (#568, #582, #584, #585)
* first-chunk cache (#580)
* sftp listdir (#571)
* http logging and fetch all (#551, #558)
* doc: entry points (#548)
* get_mapper for caching filesystems (#559)
* fix cross-device file move (#547)
* store paths without trailing "/" for DBFS (#557)
* errors that happen on _initiate_upload when closing the
AbstractBufferedFile will now be propagated (#587)
* infer_compressions with upper case suffix ($595)
* file initialiser errors (#587)
* CI fix (#563)
* local file commit cross-device (#547)
- Version 0.8.7
* fix error with pyarrow metadata for some pythons (#546)
- Version 0.8.6
* Add dbfs:// support (#504, #514)
* don't import pyarrow (#503)
* update entry points syntax (#515)
* ci precommit hooks (#534)
* random appending of a directory within the filesystems find()
method (#507, 537)
* fix git tests (#501)
* fix recursive memfs operations (#502)
* fix recorsive/maxdepth for cp (#508)
* fix listings cache timeout (#513)
* big endian bytes tests (#519)
* docs syntax (#535, 524, 520, 542)
* transactions and reads (#533)
- Don't test with distributed on python36 -- not available anymore
OBS-URL: https://build.opensuse.org/request/show/885674
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-fsspec?expand=0&rev=24
- Update to 0.8.5
Features:
* config system
* libarchive implementation
* add reference file system implementation
- Changes in version 0.8.4
Features:
* function can_be_local to see whether URL is compatible with
open_local
* concurrent cat with filecaches, if backend supports it
* jupyter FS
Fixes:
* dircache expiry after transaction
* blockcache garbage collection
* close for HDFS
* windows tests
* glob depth with "**"
- Changes in version 0.8.3
Features:
* error options for cat
* memory fs created time in detailed ls
Fixes:
* duplicate directories could appear in MemoryFileSystem
* Added support for hat dollar lbrace rbrace regex character
escapes in glob
* Fix blockcache (was doing unnecessary work)
* handle multibyte dtypes in readinto
* Fix missing kwargs in call to _copy in asyn
Other:
* Stop inheriting from pyarrow.filesystem for pyarrow>=2.0
* Raise low-level program friendly OSError.
* Guard against instance reuse in new processes
* Make hash_name a method on CachingFileSystem to make it easier
to change.
* Use get_event_loop for py3.6 compatibility
- Changes in version 0.8.2
Fixes:
* More careful strip for caching
* Version 0.8.1
Features:
* add sign to base class
* Allow calling of coroutines from normal code when running async
* Implement writing for cached many files
* Allow concurrent caching of remote files
* Add gdrive:// protocol
Fixes:
* Fix memfs with exact ls
* HTTPFileSystem requires requests and aiohttp in registry
Other:
* Allow http kwargs to clientSession
* Use extras_require in setup.py for optional dependencies
* Replacing md5 with sha256 for hash (CVE req)
* Test against Python 3.8, drop 3.5 testing
* add az alias for abfs
- Get source tarball from PyPI instead of GitHub
- BuildRequires and Suggests extra stuff and test it, where
possible
OBS-URL: https://build.opensuse.org/request/show/859069
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-fsspec?expand=0&rev=18
- Update to 0.8.0:
* Change channel order for s3fs test
* Add aiobotocore to dpes for s3fs check
* test_local: fix username not in home path (#253)
* Tighten cacheFS if dir deleted and open_local behaviour
* Add test and docs
* Attempt at context for multiple files
* #355 typo fixed
* Fix race condition of lzma import when using threads
* Small change to methods in async not to overwrite
* remove debugging print
* properly rewind MemoryFile
* Make sync functions in asyn to avoid nedsted cals to sync()
* Bail when unchaining if only one element
* Update fsspec/tests/test_api.py
* Fix local mv
* Document async, and ensure docstrings
* included smb:// in api docs
* fix smbprotocol[kerberos] docs on smb://
* smb:// wrap in SMBFileOpener instead of patching open_file
* pass glob kwargs to super class from local fs
* add test for local fs glob detail arg
* improve docs for smb protocol
* clean memfs
* fix CI failure for remote black complaining extra commas
* refactor smb functions from smv review issue
* fixed many pylint nitpicks in smb suport modules
* Remove chunk access; add mapper methods
* fixed typo "gui" => "gid"
* address review issues of smb:// protocol PR #340
OBS-URL: https://build.opensuse.org/request/show/832793
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-fsspec?expand=0&rev=16
* Fixed issues with filesystem instance caching. This was causing authorization errors in downstream libraries like gcsfs and s3fs in multi-threaded code (:pr:`155`, :pr:`181`)
* Changed the default file caching strategy to :class:`fsspec.caching.ReadAheadCache` (:pr:`193`)
* Moved file caches to the new fsspec.caching module. They're still available from their old location in fsspec.core, but we recommend using the new location for new code (:pr:`195`)
* Added a new file caching strategy, :class:`fsspec.caching.BlockCache` for fetching and caching file reads in blocks (:pr:`191`).
* Fixed equality checks for file system instance to return False when compared to objects other than file systems (:pr:`192`)
* Fixed a bug in :meth:`fsspec.FSMap.keys` returning a generator, which was consumed upon iteration (:pr:`189`).
* Removed the magic addition of aliases in AbstractFileSystem.__init__. Now alias methods are always present (:pr:`177`)
* Deprecated passing trim to :class:`fsspec.spec.AbstractBufferedFile`. Pass it in storage_options instead (:pr:`188`)
* Improved handling of requests for :class:`fsspec.implementations.http.HTTPFileSystem` when the HTTP server responds with an (incorrect) content-length of 0 (:pr:`163`)
* Added a detail=True parameter to :meth:`fsspec.spec.AbstractFileSystem.ls` (:pr:`168`)
* Fixed handling of UNC/DFS paths (:issue:`154`)
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-fsspec?expand=0&rev=12
