- Update to v2.6.1
* Security fixes for cookie handling (CVE-2022-0577 aka
bsc#1196638, GHSA-mfjm-vh54-3f96)
* Python 3.10 support
* asyncio support is no longer considered experimental, and works
out-of-the-box on Windows regardless of your Python version
* Feed exports now support pathlib.Path output paths and per-feed
item filtering and post-processing
- Remove unnecessary patches:
- remove-h2-version-restriction.patch
- add-peak-method-to-queues.patch
OBS-URL: https://build.opensuse.org/request/show/958587
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/python-Scrapy?expand=0&rev=13
- Update to 2.5.1, Security bug fix
* boo#1191446, CVE-2021-41125
* If you use HttpAuthMiddleware (i.e. the http_user and
http_pass spider attributes) for HTTP authentication,
any request exposes your credentials to the request
target.
* To prevent unintended exposure of authentication
credentials to unintended domains, you must now
additionally set a new, additional spider attribute,
http_auth_domain, and point it to the specific domain to
which the authentication credentials must be sent.
* If the http_auth_domain spider attribute is not set, the
domain of the first request will be considered the HTTP
authentication target, and authentication credentials
will only be sent in requests targeting that domain.
* If you need to send the same HTTP authentication
credentials to multiple domains, you can use
w3lib.http.basic_auth_header instead to set the value of
the Authorization header of your requests.
* If you really want your spider to send the same HTTP
authentication credentials to any domain, set the
http_auth_domain spider attribute to None.
* Finally, if you are a user of scrapy-splash, know that
this version of Scrapy breaks compatibility with
scrapy-splash 0.7.2 and earlier. You will need to upgrade
scrapy-splash to a greater version for it to continue to
work.
OBS-URL: https://build.opensuse.org/request/show/923811
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-Scrapy?expand=0&rev=21
- Update to 2.5.0:
* Official Python 3.9 support
* Experimental HTTP/2 support
* New get_retry_request() function to retry requests from spider
callbacks
* New headers_received signal that allows stopping downloads
early
* New Response.protocol attribute
- Release 2.4.1:
* Fixed feed exports overwrite support
* Fixed the asyncio event loop handling, which could make code
hang
* Fixed the IPv6-capable DNS resolver CachingHostnameResolver
for download handlers that call reactor.resolve
* Fixed the output of the genspider command showing placeholders
instead of the import part of the generated spider module
(issue 4874)
- Release 2.4.0:
* Python 3.5 support has been dropped.
* The file_path method of media pipelines can now access the
source item.
* This allows you to set a download file path based on item data.
* The new item_export_kwargs key of the FEEDS setting allows to
define keyword parameters to pass to item exporter classes.
* You can now choose whether feed exports overwrite or append to
the output file.
* For example, when using the crawl or runspider commands, you
can use the -O option instead of -o to overwrite the output
file.
* Zstd-compressed responses are now supported if zstandard is
installed.
* In settings, where the import path of a class is required, it
is now possible to pass a class object instead.
- Release 2.3.0:
* Feed exports now support Google Cloud Storage as a storage
backend
* The new FEED_EXPORT_BATCH_ITEM_COUNT setting allows to deliver
output items in batches of up to the specified number of items.
* It also serves as a workaround for delayed file delivery,
which causes Scrapy to only start item delivery after the
crawl has finished when using certain storage backends (S3,
FTP, and now GCS).
* The base implementation of item loaders has been moved into a
separate library, itemloaders, allowing usage from outside
Scrapy and a separate release schedule
- Release 2.2.1:
* The startproject command no longer makes unintended changes to
the permissions of files in the destination folder, such as
removing execution permissions.
OBS-URL: https://build.opensuse.org/request/show/889030
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-Scrapy?expand=0&rev=18
Update Scrapy to 2.2.0. NOTICE: Scrapy now requires a new package, made by the Scrapy team, called itemadapter. I have already packaged itemadapter & have submitted the package for approval. See Request 818656. Please accept python-itemadapter prior to accepting this request (otherwise build will fail). Thank you!
Changelog:
* Python 3.5.2+ is required now
* dataclass objects and attrs objects are now valid item types
* New TextResponse.json method
* New bytes_received signal that allows canceling response download
* CookiesMiddleware fixes
- Update to 2.1.0:
* New FEEDS setting to export to multiple feeds
* New Response.ip_address attribute
- Remove zope-exception-test_crawler.patch
- Add new required dependency python-itemadapter
- Omit test that fails in OBS due to https / tls issues
OBS-URL: https://build.opensuse.org/request/show/818747
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-Scrapy?expand=0&rev=16
- Update to 2.0.1:
* Python 2 support has been removed
* Partial coroutine syntax support and experimental asyncio support
* New Response.follow_all method
* FTP support for media pipelines
* New Response.certificate attribute
* IPv6 support through DNS_RESOLVER
* Response.follow_all now supports an empty URL iterable as input
* Removed top-level reactor imports to prevent errors about the wrong
Twisted reactor being installed when setting a different Twisted
reactor using TWISTED_REACTOR
- Add zope-exception-test_crawler.patch, rewriting one testcase to pass
with our version of Zope.
- Update BuildRequires based on test requirements.
OBS-URL: https://build.opensuse.org/request/show/790737
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/python-Scrapy?expand=0&rev=6
* Python 2 support has been removed
* Partial coroutine syntax support and experimental asyncio support
* New Response.follow_all method
* FTP support for media pipelines
* New Response.certificate attribute
* IPv6 support through DNS_RESOLVER
* Response.follow_all now supports an empty URL iterable as input
* Removed top-level reactor imports to prevent errors about the wrong
Twisted reactor being installed when setting a different Twisted
reactor using TWISTED_REACTOR
- Add zope-exception-test_crawler.patch, rewriting one testcase to pass
with our version of Zope.
- Update BuildRequires based on test requirements.
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-Scrapy?expand=0&rev=12
