pool/python310
SHA256
2
0
Fork 1
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Adjust CVE-2007-4559-filter-tarfile_extractall.patch.

Browse Source 
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:Factory/python310?expand=0&rev=89
This commit is contained in:
Matej Cepl 2023-05-03 14:07:47 +00:00 committed by Git OBS Bridge
parent 1ab2e0976b
commit 54a90c01cb
1 changed files with 199 additions and 177 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

372
CVE-2007-4559-filter-tarfile_extractall.patch
View File

@ -1,25 +1,8 @@
From cde089c808a2c21dd311905ba7f1b7e1004c0ada Mon Sep 17 00:00:00 2001 diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst
From: Petr Viktorin <encukou@gmail.com> index 311aae414ae..3864e03898d 100644
Date: Tue, 31 Jan 2023 14:40:52 +0100
Subject: [PATCH 01/15] =?UTF-8?q?Implement=20PEP=20706=20=E2=80=93=20Filte?=
=?UTF-8?q?r=20for=20tarfile.extractall?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
---
Doc/library/shutil.rst | 24
Doc/library/tarfile.rst | 457 ++++
Lib/shutil.py | 17
Lib/tarfile.py | 361 +++
Lib/test/test_shutil.py | 41
Lib/test/test_tarfile.py | 947 +++++++++-
Misc/NEWS.d/next/Library/2023-03-23-15-24-38.gh-issue-102953.YR4KaK.rst | 4
7 files changed, 1753 insertions(+), 98 deletions(-)
--- a/Doc/library/shutil.rst --- a/Doc/library/shutil.rst
+++ b/Doc/library/shutil.rst +++ b/Doc/library/shutil.rst
@@ -620,7 +620,7 @@ provided. They rely on the :mod:`zipfil @@ -620,7 +620,7 @@ provided. They rely on the :mod:`zipfile` and :mod:`tarfile` modules.
Remove the archive format *name* from the list of supported formats. Remove the archive format *name* from the list of supported formats.
@ -28,12 +11,13 @@ Content-Transfer-Encoding: 8bit
Unpack an archive. *filename* is the full path of the archive. Unpack an archive. *filename* is the full path of the archive.
@@ -634,6 +634,14 @@ provided. They rely on the :mod:`zipfil @@ -634,6 +634,15 @@ provided. They rely on the :mod:`zipfile` and :mod:`tarfile` modules.
registered for that extension. In case none is found, registered for that extension. In case none is found,
a :exc:`ValueError` is raised. a :exc:`ValueError` is raised.
+ The keyword-only *filter* argument is passed to the underlying unpacking + The keyword-only *filter* argument, which was added in Python 3.11.4,
+ function. For zip files, *filter* is not accepted. + is passed to the underlying unpacking function.
+ For zip files, *filter* is not accepted.
+ For tar files, it is recommended to set it to ``'data'``, + For tar files, it is recommended to set it to ``'data'``,
+ unless using features specific to tar and UNIX-like filesystems. + unless using features specific to tar and UNIX-like filesystems.
+ (See :ref:`tarfile-extraction-filter` for details.) + (See :ref:`tarfile-extraction-filter` for details.)
@ -43,26 +27,27 @@ Content-Transfer-Encoding: 8bit
.. audit-event:: shutil.unpack_archive filename,extract_dir,format shutil.unpack_archive .. audit-event:: shutil.unpack_archive filename,extract_dir,format shutil.unpack_archive
.. warning:: .. warning::
@@ -646,6 +654,9 @@ provided. They rely on the :mod:`zipfil @@ -646,6 +655,9 @@ provided. They rely on the :mod:`zipfile` and :mod:`tarfile` modules.
.. versionchanged:: 3.7 .. versionchanged:: 3.7
Accepts a :term:`path-like object` for *filename* and *extract_dir*. Accepts a :term:`path-like object` for *filename* and *extract_dir*.
+ .. versionchanged:: 3.12 + .. versionchanged:: 3.11.4
+ Added the *filter* argument. + Added the *filter* argument.
+ +
.. function:: register_unpack_format(name, extensions, function[, extra_args[, description]]) .. function:: register_unpack_format(name, extensions, function[, extra_args[, description]])
Registers an unpack format. *name* is the name of the format and Registers an unpack format. *name* is the name of the format and
@@ -653,11 +664,14 @@ provided. They rely on the :mod:`zipfil @@ -653,11 +665,14 @@ provided. They rely on the :mod:`zipfile` and :mod:`tarfile` modules.
``.zip`` for Zip files. ``.zip`` for Zip files.
*function* is the callable that will be used to unpack archives. The *function* is the callable that will be used to unpack archives. The
- callable will receive the path of the archive, followed by the directory - callable will receive the path of the archive, followed by the directory
- the archive must be extracted to. - the archive must be extracted to.
+ callable will receive: -
- When provided, *extra_args* is a sequence of ``(name, value)`` tuples that - When provided, *extra_args* is a sequence of ``(name, value)`` tuples that
- will be passed as keywords arguments to the callable. - will be passed as keywords arguments to the callable.
+ callable will receive:
+
+ - the path of the archive, as a positional argument; + - the path of the archive, as a positional argument;
+ - the directory the archive must be extracted to, as a positional argument; + - the directory the archive must be extracted to, as a positional argument;
+ - possibly a *filter* keyword argument, if it was given to + - possibly a *filter* keyword argument, if it was given to
@ -72,23 +57,11 @@ Content-Transfer-Encoding: 8bit
*description* can be provided to describe the format, and will be returned *description* can be provided to describe the format, and will be returned
by the :func:`get_unpack_formats` function. by the :func:`get_unpack_formats` function.
diff --git a/Doc/library/tarfile.rst b/Doc/library/tarfile.rst
index 226513f5fc1..836444ebb34 100644
--- a/Doc/library/tarfile.rst --- a/Doc/library/tarfile.rst
+++ b/Doc/library/tarfile.rst +++ b/Doc/library/tarfile.rst
@@ -36,6 +36,13 @@ Some facts and figures: @@ -206,6 +206,38 @@ The :mod:`tarfile` module defines the following exceptions:
.. versionchanged:: 3.3
Added support for :mod:`lzma` compression.
+.. versionchanged:: 3.12
+ Archives are extracted using a :ref:`filter <tarfile-extraction-filter>`,
+ which makes it possible to either limit surprising/dangerous features,
+ or to acknowledge that they are expected and the archive is fully trusted.
+ By default, archives are fully trusted, but this default is deprecated
+ and slated to change in Python 3.14.
+
.. function:: open(name=None, mode='r', fileobj=None, bufsize=10240, **kwargs)
@@ -206,6 +213,38 @@ The :mod:`tarfile` module defines the fo
Is raised by :meth:`TarInfo.frombuf` if the buffer it gets is invalid. Is raised by :meth:`TarInfo.frombuf` if the buffer it gets is invalid.
@ -127,7 +100,7 @@ Content-Transfer-Encoding: 8bit
The following constants are available at the module level: The following constants are available at the module level:
.. data:: ENCODING .. data:: ENCODING
@@ -316,11 +355,8 @@ be finalized; only the internally used f @@ -316,11 +348,8 @@ be finalized; only the internally used file object will be closed. See the
*debug* can be set from ``0`` (no debug messages) up to ``3`` (all debug *debug* can be set from ``0`` (no debug messages) up to ``3`` (all debug
messages). The messages are written to ``sys.stderr``. messages). The messages are written to ``sys.stderr``.
@ -141,7 +114,7 @@ Content-Transfer-Encoding: 8bit
The *encoding* and *errors* arguments define the character encoding to be The *encoding* and *errors* arguments define the character encoding to be
used for reading or writing the archive and how conversion errors are going used for reading or writing the archive and how conversion errors are going
@@ -387,7 +423,7 @@ be finalized; only the internally used f @@ -387,7 +416,7 @@ be finalized; only the internally used file object will be closed. See the
available. available.
@ -150,12 +123,12 @@ Content-Transfer-Encoding: 8bit
Extract all members from the archive to the current working directory or Extract all members from the archive to the current working directory or
directory *path*. If optional *members* is given, it must be a subset of the directory *path*. If optional *members* is given, it must be a subset of the
@@ -401,6 +437,12 @@ be finalized; only the internally used f @@ -401,6 +430,12 @@ be finalized; only the internally used file object will be closed. See the
are used to set the owner/group for the extracted files. Otherwise, the named are used to set the owner/group for the extracted files. Otherwise, the named
values from the tarfile are used. values from the tarfile are used.
+ The *filter* argument specifies how ``members`` are modified or rejected + The *filter* argument, which was added in Python 3.11.4, specifies how
+ before extraction. + ``members`` are modified or rejected before extraction.
+ See :ref:`tarfile-extraction-filter` for details. + See :ref:`tarfile-extraction-filter` for details.
+ It is recommended to set this explicitly depending on which *tar* features + It is recommended to set this explicitly depending on which *tar* features
+ you need to support. + you need to support.
@ -163,7 +136,7 @@ Content-Transfer-Encoding: 8bit
.. warning:: .. warning::
Never extract archives from untrusted sources without prior inspection. Never extract archives from untrusted sources without prior inspection.
@@ -408,14 +450,20 @@ be finalized; only the internally used f @@ -408,14 +443,20 @@ be finalized; only the internally used file object will be closed. See the
that have absolute filenames starting with ``"/"`` or filenames with two that have absolute filenames starting with ``"/"`` or filenames with two
dots ``".."``. dots ``".."``.
@ -176,7 +149,7 @@ Content-Transfer-Encoding: 8bit
.. versionchanged:: 3.6 .. versionchanged:: 3.6
The *path* parameter accepts a :term:`path-like object`. The *path* parameter accepts a :term:`path-like object`.
+ .. versionchanged:: 3.12 + .. versionchanged:: 3.11.4
+ Added the *filter* parameter. + Added the *filter* parameter.
+ +
@ -185,7 +158,7 @@ Content-Transfer-Encoding: 8bit
Extract a member from the archive to the current working directory, using its Extract a member from the archive to the current working directory, using its
full name. Its file information is extracted as accurately as possible. *member* full name. Its file information is extracted as accurately as possible. *member*
@@ -423,9 +471,8 @@ be finalized; only the internally used f @@ -423,9 +464,8 @@ be finalized; only the internally used file object will be closed. See the
directory using *path*. *path* may be a :term:`path-like object`. directory using *path*. *path* may be a :term:`path-like object`.
File attributes (owner, mtime, mode) are set unless *set_attrs* is false. File attributes (owner, mtime, mode) are set unless *set_attrs* is false.
@ -197,7 +170,7 @@ Content-Transfer-Encoding: 8bit
.. note:: .. note::
@@ -436,6 +483,9 @@ be finalized; only the internally used f @@ -436,6 +476,9 @@ be finalized; only the internally used file object will be closed. See the
See the warning for :meth:`extractall`. See the warning for :meth:`extractall`.
@ -207,17 +180,17 @@ Content-Transfer-Encoding: 8bit
.. versionchanged:: 3.2 .. versionchanged:: 3.2
Added the *set_attrs* parameter. Added the *set_attrs* parameter.
@@ -445,6 +495,9 @@ be finalized; only the internally used f @@ -445,6 +488,9 @@ be finalized; only the internally used file object will be closed. See the
.. versionchanged:: 3.6 .. versionchanged:: 3.6
The *path* parameter accepts a :term:`path-like object`. The *path* parameter accepts a :term:`path-like object`.
+ .. versionchanged:: 3.12 + .. versionchanged:: 3.11.4
+ Added the *filter* parameter. + Added the *filter* parameter.
+ +
.. method:: TarFile.extractfile(member) .. method:: TarFile.extractfile(member)
@@ -457,6 +510,55 @@ be finalized; only the internally used f @@ -457,6 +503,57 @@ be finalized; only the internally used file object will be closed. See the
.. versionchanged:: 3.3 .. versionchanged:: 3.3
Return an :class:`io.BufferedReader` object. Return an :class:`io.BufferedReader` object.
@ -244,7 +217,7 @@ Content-Transfer-Encoding: 8bit
+ +
+.. attribute:: TarFile.extraction_filter +.. attribute:: TarFile.extraction_filter
+ +
+ .. versionadded:: 3.12 + .. versionadded:: 3.11.4
+ +
+ The :ref:`extraction filter <tarfile-extraction-filter>` used + The :ref:`extraction filter <tarfile-extraction-filter>` used
+ as a default for the *filter* argument of :meth:`~TarFile.extract` + as a default for the *filter* argument of :meth:`~TarFile.extract`
@ -255,10 +228,12 @@ Content-Transfer-Encoding: 8bit
+ argument to :meth:`~TarFile.extract`. + argument to :meth:`~TarFile.extract`.
+ +
+ If ``extraction_filter`` is ``None`` (the default), + If ``extraction_filter`` is ``None`` (the default),
+ calling an extraction method without a *filter* argument will raise a + calling an extraction method without a *filter* argument will
+ ``DeprecationWarning``, + use the :func:`fully_trusted <fully_trusted_filter>` filter for
+ and fall back to the :func:`fully_trusted <fully_trusted_filter>` filter, + compatibility with previous Python versions.
+ whose dangerous behavior matches previous versions of Python. +
+ In Python 3.12+, leaving ``extraction_filter=None`` will emit a
+ ``DeprecationWarning``.
+ +
+ In Python 3.14+, leaving ``extraction_filter=None`` will cause + In Python 3.14+, leaving ``extraction_filter=None`` will cause
+ extraction methods to use the :func:`data <data_filter>` filter by default. + extraction methods to use the :func:`data <data_filter>` filter by default.
@ -273,14 +248,14 @@ Content-Transfer-Encoding: 8bit
.. method:: TarFile.add(name, arcname=None, recursive=True, *, filter=None) .. method:: TarFile.add(name, arcname=None, recursive=True, *, filter=None)
@@ -532,8 +634,23 @@ permissions, owner etc.), it provides so @@ -532,7 +629,27 @@ permissions, owner etc.), it provides some useful methods to determine its type.
It does *not* contain the file's data itself. It does *not* contain the file's data itself.
:class:`TarInfo` objects are returned by :class:`TarFile`'s methods :class:`TarInfo` objects are returned by :class:`TarFile`'s methods
-:meth:`getmember`, :meth:`getmembers` and :meth:`gettarinfo`. -:meth:`getmember`, :meth:`getmembers` and :meth:`gettarinfo`.
+:meth:`~TarFile.getmember`, :meth:`~TarFile.getmembers` and +:meth:`~TarFile.getmember`, :meth:`~TarFile.getmembers` and
+:meth:`~TarFile.gettarinfo`. +:meth:`~TarFile.gettarinfo`.
+
+Modifying the objects returned by :meth:`~!TarFile.getmember` or +Modifying the objects returned by :meth:`~!TarFile.getmember` or
+:meth:`~!TarFile.getmembers` will affect all subsequent +:meth:`~!TarFile.getmembers` will affect all subsequent
+operations on the archive. +operations on the archive.
@ -295,10 +270,14 @@ Content-Transfer-Encoding: 8bit
+ ignore the corresponding metadata, leaving it set to a default. + ignore the corresponding metadata, leaving it set to a default.
+- :meth:`~TarFile.addfile` will fail. +- :meth:`~TarFile.addfile` will fail.
+- :meth:`~TarFile.list` will print a placeholder string. +- :meth:`~TarFile.list` will print a placeholder string.
+
+
+.. versionchanged:: 3.11.4
+ Added :meth:`~TarInfo.replace` and handling of ``None``.
.. class:: TarInfo(name="") .. class:: TarInfo(name="")
@@ -566,24 +683,39 @@ A ``TarInfo`` object has the following public data attributes:
@@ -566,24 +683,39 @@ A ``TarInfo`` object has the following p
.. attribute:: TarInfo.name .. attribute:: TarInfo.name
@ -320,7 +299,7 @@ Content-Transfer-Encoding: 8bit
+ as in :attr:`os.stat_result.st_mtime`. + as in :attr:`os.stat_result.st_mtime`.
- Time of last modification. - Time of last modification.
+ .. versionchanged:: 3.12 + .. versionchanged:: 3.11.4
+ Can be set to ``None`` for :meth:`~TarFile.extract` and + Can be set to ``None`` for :meth:`~TarFile.extract` and
+ :meth:`~TarFile.extractall`, causing extraction to skip applying this + :meth:`~TarFile.extractall`, causing extraction to skip applying this
@ -332,7 +311,7 @@ Content-Transfer-Encoding: 8bit
- Permission bits. - Permission bits.
+ Permission bits, as for :func:`os.chmod`. + Permission bits, as for :func:`os.chmod`.
+ .. versionchanged:: 3.12 + .. versionchanged:: 3.11.4
+ +
+ Can be set to ``None`` for :meth:`~TarFile.extract` and + Can be set to ``None`` for :meth:`~TarFile.extract` and
+ :meth:`~TarFile.extractall`, causing extraction to skip applying this + :meth:`~TarFile.extractall`, causing extraction to skip applying this
@ -340,7 +319,7 @@ Content-Transfer-Encoding: 8bit
.. attribute:: TarInfo.type .. attribute:: TarInfo.type
@@ -595,35 +727,76 @@ A ``TarInfo`` object has the following p @@ -595,35 +727,76 @@ A ``TarInfo`` object has the following public data attributes:
.. attribute:: TarInfo.linkname .. attribute:: TarInfo.linkname
@ -355,7 +334,7 @@ Content-Transfer-Encoding: 8bit
User ID of the user who originally stored this member. User ID of the user who originally stored this member.
+ .. versionchanged:: 3.12 + .. versionchanged:: 3.11.4
+ +
+ Can be set to ``None`` for :meth:`~TarFile.extract` and + Can be set to ``None`` for :meth:`~TarFile.extract` and
+ :meth:`~TarFile.extractall`, causing extraction to skip applying this + :meth:`~TarFile.extractall`, causing extraction to skip applying this
@ -366,7 +345,7 @@ Content-Transfer-Encoding: 8bit
Group ID of the user who originally stored this member. Group ID of the user who originally stored this member.
+ .. versionchanged:: 3.12 + .. versionchanged:: 3.11.4
+ +
+ Can be set to ``None`` for :meth:`~TarFile.extract` and + Can be set to ``None`` for :meth:`~TarFile.extract` and
+ :meth:`~TarFile.extractall`, causing extraction to skip applying this + :meth:`~TarFile.extractall`, causing extraction to skip applying this
@ -377,7 +356,7 @@ Content-Transfer-Encoding: 8bit
User name. User name.
+ .. versionchanged:: 3.12 + .. versionchanged:: 3.11.4
+ +
+ Can be set to ``None`` for :meth:`~TarFile.extract` and + Can be set to ``None`` for :meth:`~TarFile.extract` and
+ :meth:`~TarFile.extractall`, causing extraction to skip applying this + :meth:`~TarFile.extractall`, causing extraction to skip applying this
@ -388,7 +367,7 @@ Content-Transfer-Encoding: 8bit
Group name. Group name.
+ .. versionchanged:: 3.12 + .. versionchanged:: 3.11.4
+ +
+ Can be set to ``None`` for :meth:`~TarFile.extract` and + Can be set to ``None`` for :meth:`~TarFile.extract` and
+ :meth:`~TarFile.extractall`, causing extraction to skip applying this + :meth:`~TarFile.extractall`, causing extraction to skip applying this
@ -403,7 +382,7 @@ Content-Transfer-Encoding: 8bit
+ uid=..., gid=..., uname=..., gname=..., + uid=..., gid=..., uname=..., gname=...,
+ deep=True) + deep=True)
+ +
+ .. versionadded:: 3.12 + .. versionadded:: 3.11.4
+ +
+ Return a *new* copy of the :class:`!TarInfo` object with the given attributes + Return a *new* copy of the :class:`!TarInfo` object with the given attributes
+ changed. For example, to return a ``TarInfo`` with the group name set to + changed. For example, to return a ``TarInfo`` with the group name set to
@ -417,7 +396,7 @@ Content-Transfer-Encoding: 8bit
A :class:`TarInfo` object also provides some convenient query methods: A :class:`TarInfo` object also provides some convenient query methods:
@@ -673,9 +846,258 @@ A :class:`TarInfo` object also provides @@ -673,9 +846,259 @@ A :class:`TarInfo` object also provides some convenient query methods:
Return :const:`True` if it is one of character device, block device or FIFO. Return :const:`True` if it is one of character device, block device or FIFO.
@ -426,7 +405,7 @@ Content-Transfer-Encoding: 8bit
+Extraction filters +Extraction filters
+------------------ +------------------
+ +
+.. versionadded:: 3.12 +.. versionadded:: 3.11.4
+ +
+The *tar* format is designed to capture all details of a UNIX-like filesystem, +The *tar* format is designed to capture all details of a UNIX-like filesystem,
+which makes it very powerful. +which makes it very powerful.
@ -463,9 +442,10 @@ Content-Transfer-Encoding: 8bit
+ +
+* ``None`` (default): Use :attr:`TarFile.extraction_filter`. +* ``None`` (default): Use :attr:`TarFile.extraction_filter`.
+ +
+ If that is also ``None`` (the default), raise a ``DeprecationWarning``, + If that is also ``None`` (the default), the ``'fully_trusted'``
+ and fall back to the ``'fully_trusted'`` filter, whose dangerous behavior + filter will be used (for compatibility with earlier versions of Python).
+ matches previous versions of Python. +
+ In Python 3.12, the default will emit a ``DeprecationWarning``.
+ +
+ In Python 3.14, the ``'data'`` filter will become the default instead. + In Python 3.14, the ``'data'`` filter will become the default instead.
+ It's possible to switch earlier; see :attr:`TarFile.extraction_filter`. + It's possible to switch earlier; see :attr:`TarFile.extraction_filter`.
@ -602,7 +582,7 @@ Content-Transfer-Encoding: 8bit
+Supporting older Python versions +Supporting older Python versions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ +
+Extraction filters were added to Python 3.12, but may be backported to older +Extraction filters were added to Python 3.12, and are backported to older
+versions as security updates. +versions as security updates.
+To check whether the feature is available, use e.g. +To check whether the feature is available, use e.g.
+``hasattr(tarfile, 'data_filter')`` rather than checking the Python version. +``hasattr(tarfile, 'data_filter')`` rather than checking the Python version.
@ -676,7 +656,7 @@ Content-Transfer-Encoding: 8bit
Command-Line Interface Command-Line Interface
---------------------- ----------------------
@@ -745,6 +1167,13 @@ Command-line options @@ -745,6 +1168,15 @@ Command-line options
Verbose output. Verbose output.
@ -686,22 +666,41 @@ Content-Transfer-Encoding: 8bit
+ See :ref:`tarfile-extraction-filter` for details. + See :ref:`tarfile-extraction-filter` for details.
+ Only string names are accepted (that is, ``fully_trusted``, ``tar``, + Only string names are accepted (that is, ``fully_trusted``, ``tar``,
+ and ``data``). + and ``data``).
+
+ .. versionadded:: 3.11.4
+ +
.. _tar-examples: .. _tar-examples:
Examples Examples
@@ -754,7 +1183,7 @@ How to extract an entire tar archive to diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst
index 47e38ae76ba..43da72aece9 100644
import tarfile --- a/Doc/whatsnew/3.10.rst
tar = tarfile.open("sample.tar.gz") +++ b/Doc/whatsnew/3.10.rst
- tar.extractall() @@ -2332,3 +2332,19 @@ The deprecated :mod:`mailcap` module now refuses to inject unsafe text
+ tar.extractall(filter='data') text, it will warn and act as if a match was not found (or for test commands,
tar.close() as if the test failed).
(Contributed by Petr Viktorin in :gh:`98966`.)
How to extract a subset of a tar archive with :meth:`TarFile.extractall` using +
+Notable Changes in 3.10.12
+==========================
+
+tarfile
+-------
+
+* The extraction methods in :mod:`tarfile`, and :func:`shutil.unpack_archive`,
+ have a new a *filter* argument that allows limiting tar features than may be
+ surprising or dangerous, such as creating files outside the destination
+ directory.
+ See :ref:`tarfile-extraction-filter` for details.
+ In Python 3.12, use without the *filter* argument will show a
+ :exc:`DeprecationWarning`.
+ In Python 3.14, the default will switch to ``'data'``.
+ (Contributed by Petr Viktorin in :pep:`706`.)
diff --git a/Lib/shutil.py b/Lib/shutil.py
index b7bffa3ea41..482ce95a7b2 100644
--- a/Lib/shutil.py --- a/Lib/shutil.py
+++ b/Lib/shutil.py +++ b/Lib/shutil.py
@@ -1222,7 +1222,7 @@ def _unpack_zipfile(filename, extract_di @@ -1222,7 +1222,7 @@ def _unpack_zipfile(filename, extract_dir):
finally: finally:
zip.close() zip.close()
@ -710,7 +709,7 @@ Content-Transfer-Encoding: 8bit
"""Unpack tar/tar.gz/tar.bz2/tar.xz `filename` to `extract_dir` """Unpack tar/tar.gz/tar.bz2/tar.xz `filename` to `extract_dir`
""" """
import tarfile # late import for breaking circular dependency import tarfile # late import for breaking circular dependency
@@ -1232,7 +1232,7 @@ def _unpack_tarfile(filename, extract_di @@ -1232,7 +1232,7 @@ def _unpack_tarfile(filename, extract_dir):
raise ReadError( raise ReadError(
"%s is not a compressed or uncompressed tar file" % filename) "%s is not a compressed or uncompressed tar file" % filename)
try: try:
@ -728,7 +727,7 @@ Content-Transfer-Encoding: 8bit
"""Unpack an archive. """Unpack an archive.
`filename` is the name of the archive. `filename` is the name of the archive.
@@ -1279,6 +1279,9 @@ def unpack_archive(filename, extract_dir @@ -1279,6 +1279,9 @@ def unpack_archive(filename, extract_dir=None, format=None):
was registered for that extension. was registered for that extension.
In case none is found, a ValueError is raised. In case none is found, a ValueError is raised.
@ -738,7 +737,7 @@ Content-Transfer-Encoding: 8bit
""" """
sys.audit("shutil.unpack_archive", filename, extract_dir, format) sys.audit("shutil.unpack_archive", filename, extract_dir, format)
@@ -1288,6 +1291,10 @@ def unpack_archive(filename, extract_dir @@ -1288,6 +1291,10 @@ def unpack_archive(filename, extract_dir=None, format=None):
extract_dir = os.fspath(extract_dir) extract_dir = os.fspath(extract_dir)
filename = os.fspath(filename) filename = os.fspath(filename)
@ -749,7 +748,7 @@ Content-Transfer-Encoding: 8bit
if format is not None: if format is not None:
try: try:
format_info = _UNPACK_FORMATS[format] format_info = _UNPACK_FORMATS[format]
@@ -1295,7 +1302,7 @@ def unpack_archive(filename, extract_dir @@ -1295,7 +1302,7 @@ def unpack_archive(filename, extract_dir=None, format=None):
raise ValueError("Unknown unpack format '{0}'".format(format)) from None raise ValueError("Unknown unpack format '{0}'".format(format)) from None
func = format_info[1] func = format_info[1]
@ -758,7 +757,7 @@ Content-Transfer-Encoding: 8bit
else: else:
# we need to look at the registered unpackers supported extensions # we need to look at the registered unpackers supported extensions
format = _find_unpack_format(filename) format = _find_unpack_format(filename)
@@ -1303,7 +1310,7 @@ def unpack_archive(filename, extract_dir @@ -1303,7 +1310,7 @@ def unpack_archive(filename, extract_dir=None, format=None):
raise ReadError("Unknown archive format '{0}'".format(filename)) raise ReadError("Unknown archive format '{0}'".format(filename))
func = _UNPACK_FORMATS[format][1] func = _UNPACK_FORMATS[format][1]
@ -767,9 +766,11 @@ Content-Transfer-Encoding: 8bit
func(filename, extract_dir, **kwargs) func(filename, extract_dir, **kwargs)
diff --git a/Lib/tarfile.py b/Lib/tarfile.py
index dea150e8dbb..40599f27bce 100755
--- a/Lib/tarfile.py --- a/Lib/tarfile.py
+++ b/Lib/tarfile.py +++ b/Lib/tarfile.py
@@ -46,6 +46,7 @@ import time @@ -46,6 +46,7 @@
import struct import struct
import copy import copy
import re import re
@ -777,20 +778,15 @@ Content-Transfer-Encoding: 8bit
try: try:
import pwd import pwd
@@ -69,7 +70,11 @@ except NameError: @@ -71,6 +72,7 @@
__all__ = ["TarFile", "TarInfo", "is_tarfile", "TarError", "ReadError",
"CompressionError", "StreamError", "ExtractError", "HeaderError",
"ENCODING", "USTAR_FORMAT", "GNU_FORMAT", "PAX_FORMAT", "ENCODING", "USTAR_FORMAT", "GNU_FORMAT", "PAX_FORMAT",
- "DEFAULT_FORMAT", "open"] "DEFAULT_FORMAT", "open"]
+ "DEFAULT_FORMAT", "open","fully_trusted_filter", "data_filter",
+ "tar_filter", "FilterError", "AbsoluteLinkError",
+ "OutsideDestinationError", "SpecialFileError", "AbsolutePathError",
+ "LinkOutsideDestinationError"]
+
+
#--------------------------------------------------------- #---------------------------------------------------------
# tar constants # tar constants
@@ -158,6 +163,8 @@ else: #---------------------------------------------------------
@@ -158,6 +160,8 @@
def stn(s, length, encoding, errors): def stn(s, length, encoding, errors):
"""Convert a string to a null-terminated bytes object. """Convert a string to a null-terminated bytes object.
""" """
@ -799,7 +795,7 @@ Content-Transfer-Encoding: 8bit
s = s.encode(encoding, errors) s = s.encode(encoding, errors)
return s[:length] + (length - len(s)) * NUL return s[:length] + (length - len(s)) * NUL
@@ -709,9 +716,127 @@ class ExFileObject(io.BufferedReader): @@ -709,9 +713,127 @@ def __init__(self, tarfile, tarinfo):
super().__init__(fileobj) super().__init__(fileobj)
#class ExFileObject #class ExFileObject
@ -927,7 +923,7 @@ Content-Transfer-Encoding: 8bit
class TarInfo(object): class TarInfo(object):
"""Informational class which holds the details about an """Informational class which holds the details about an
archive member given by a tar header block. archive member given by a tar header block.
@@ -792,12 +917,44 @@ class TarInfo(object): @@ -792,12 +914,44 @@ def linkpath(self, linkname):
def __repr__(self): def __repr__(self):
return "<%s %r at %#x>" % (self.__class__.__name__,self.name,id(self)) return "<%s %r at %#x>" % (self.__class__.__name__,self.name,id(self))
@ -973,7 +969,7 @@ Content-Transfer-Encoding: 8bit
"uid": self.uid, "uid": self.uid,
"gid": self.gid, "gid": self.gid,
"size": self.size, "size": self.size,
@@ -820,6 +977,9 @@ class TarInfo(object): @@ -820,6 +974,9 @@ def tobuf(self, format=DEFAULT_FORMAT, encoding=ENCODING, errors="surrogateescap
"""Return a tar header as a string of 512 byte blocks. """Return a tar header as a string of 512 byte blocks.
""" """
info = self.get_info() info = self.get_info()
@ -983,7 +979,7 @@ Content-Transfer-Encoding: 8bit
if format == USTAR_FORMAT: if format == USTAR_FORMAT:
return self.create_ustar_header(info, encoding, errors) return self.create_ustar_header(info, encoding, errors)
@@ -950,6 +1110,12 @@ class TarInfo(object): @@ -950,6 +1107,12 @@ def _create_header(info, format, encoding, errors):
devmajor = stn("", 8, encoding, errors) devmajor = stn("", 8, encoding, errors)
devminor = stn("", 8, encoding, errors) devminor = stn("", 8, encoding, errors)
@ -996,7 +992,7 @@ Content-Transfer-Encoding: 8bit
parts = [ parts = [
stn(info.get("name", ""), 100, encoding, errors), stn(info.get("name", ""), 100, encoding, errors),
itn(info.get("mode", 0) & 0o7777, 8, format), itn(info.get("mode", 0) & 0o7777, 8, format),
@@ -958,7 +1124,7 @@ class TarInfo(object): @@ -958,7 +1121,7 @@ def _create_header(info, format, encoding, errors):
itn(info.get("size", 0), 12, format), itn(info.get("size", 0), 12, format),
itn(info.get("mtime", 0), 12, format), itn(info.get("mtime", 0), 12, format),
b" ", # checksum field b" ", # checksum field
@ -1005,7 +1001,7 @@ Content-Transfer-Encoding: 8bit
stn(info.get("linkname", ""), 100, encoding, errors), stn(info.get("linkname", ""), 100, encoding, errors),
info.get("magic", POSIX_MAGIC), info.get("magic", POSIX_MAGIC),
stn(info.get("uname", ""), 32, encoding, errors), stn(info.get("uname", ""), 32, encoding, errors),
@@ -1468,6 +1634,8 @@ class TarFile(object): @@ -1468,6 +1631,8 @@ class TarFile(object):
fileobject = ExFileObject # The file-object for extractfile(). fileobject = ExFileObject # The file-object for extractfile().
@ -1014,7 +1010,7 @@ Content-Transfer-Encoding: 8bit
def __init__(self, name=None, mode="r", fileobj=None, format=None, def __init__(self, name=None, mode="r", fileobj=None, format=None,
tarinfo=None, dereference=None, ignore_zeros=None, encoding=None, tarinfo=None, dereference=None, ignore_zeros=None, encoding=None,
errors="surrogateescape", pax_headers=None, debug=None, errors="surrogateescape", pax_headers=None, debug=None,
@@ -1940,7 +2108,10 @@ class TarFile(object): @@ -1940,7 +2105,10 @@ def list(self, verbose=True, *, members=None):
members = self members = self
for tarinfo in members: for tarinfo in members:
if verbose: if verbose:
@ -1026,7 +1022,7 @@ Content-Transfer-Encoding: 8bit
_safe_print("%s/%s" % (tarinfo.uname or tarinfo.uid, _safe_print("%s/%s" % (tarinfo.uname or tarinfo.uid,
tarinfo.gname or tarinfo.gid)) tarinfo.gname or tarinfo.gid))
if tarinfo.ischr() or tarinfo.isblk(): if tarinfo.ischr() or tarinfo.isblk():
@@ -1948,8 +2119,11 @@ class TarFile(object): @@ -1948,8 +2116,11 @@ def list(self, verbose=True, *, members=None):
("%d,%d" % (tarinfo.devmajor, tarinfo.devminor))) ("%d,%d" % (tarinfo.devmajor, tarinfo.devminor)))
else: else:
_safe_print("%10d" % tarinfo.size) _safe_print("%10d" % tarinfo.size)
@ -1040,7 +1036,7 @@ Content-Transfer-Encoding: 8bit
_safe_print(tarinfo.name + ("/" if tarinfo.isdir() else "")) _safe_print(tarinfo.name + ("/" if tarinfo.isdir() else ""))
@@ -2036,32 +2210,63 @@ class TarFile(object): @@ -2036,32 +2207,58 @@ def addfile(self, tarinfo, fileobj=None):
self.members.append(tarinfo) self.members.append(tarinfo)
@ -1049,11 +1045,6 @@ Content-Transfer-Encoding: 8bit
+ if filter is None: + if filter is None:
+ filter = self.extraction_filter + filter = self.extraction_filter
+ if filter is None: + if filter is None:
+ warnings.warn(
+ 'Python 3.14 will, by default, filter extracted tar '
+ + 'archives and reject files or modify their metadata. '
+ + 'Use the filter argument to control this behavior.',
+ DeprecationWarning)
+ return fully_trusted_filter + return fully_trusted_filter
+ if isinstance(filter, str): + if isinstance(filter, str):
+ raise TypeError( + raise TypeError(
@ -1114,7 +1105,7 @@ Content-Transfer-Encoding: 8bit
# Set correct owner, mtime and filemode on directories. # Set correct owner, mtime and filemode on directories.
for tarinfo in directories: for tarinfo in directories:
@@ -2071,12 +2276,10 @@ class TarFile(object): @@ -2071,12 +2268,10 @@ def extractall(self, path=".", members=None, *, numeric_owner=False):
self.utime(tarinfo, dirpath) self.utime(tarinfo, dirpath)
self.chmod(tarinfo, dirpath) self.chmod(tarinfo, dirpath)
except ExtractError as e: except ExtractError as e:
@ -1130,7 +1121,7 @@ Content-Transfer-Encoding: 8bit
"""Extract a member from the archive to the current working directory, """Extract a member from the archive to the current working directory,
using its full name. Its file information is extracted as accurately using its full name. Its file information is extracted as accurately
as possible. `member' may be a filename or a TarInfo object. You can as possible. `member' may be a filename or a TarInfo object. You can
@@ -2084,35 +2287,70 @@ class TarFile(object): @@ -2084,35 +2279,70 @@ def extract(self, member, path="", set_attrs=True, *, numeric_owner=False):
mtime, mode) are set unless `set_attrs' is False. If `numeric_owner` mtime, mode) are set unless `set_attrs' is False. If `numeric_owner`
is True, only the numbers for user/group names are used and not is True, only the numbers for user/group names are used and not
the names. the names.
@ -1212,7 +1203,7 @@ Content-Transfer-Encoding: 8bit
def extractfile(self, member): def extractfile(self, member):
"""Extract a member from the archive as a file object. `member' may be """Extract a member from the archive as a file object. `member' may be
@@ -2199,9 +2437,13 @@ class TarFile(object): @@ -2199,9 +2429,13 @@ def makedir(self, tarinfo, targetpath):
"""Make a directory called targetpath. """Make a directory called targetpath.
""" """
try: try:
@ -1229,7 +1220,7 @@ Content-Transfer-Encoding: 8bit
except FileExistsError: except FileExistsError:
pass pass
@@ -2244,6 +2486,9 @@ class TarFile(object): @@ -2244,6 +2478,9 @@ def makedev(self, tarinfo, targetpath):
raise ExtractError("special devices not supported by system") raise ExtractError("special devices not supported by system")
mode = tarinfo.mode mode = tarinfo.mode
@ -1239,7 +1230,7 @@ Content-Transfer-Encoding: 8bit
if tarinfo.isblk(): if tarinfo.isblk():
mode |= stat.S_IFBLK mode |= stat.S_IFBLK
else: else:
@@ -2265,7 +2510,6 @@ class TarFile(object): @@ -2265,7 +2502,6 @@ def makelink(self, tarinfo, targetpath):
os.unlink(targetpath) os.unlink(targetpath)
os.symlink(tarinfo.linkname, targetpath) os.symlink(tarinfo.linkname, targetpath)
else: else:
@ -1247,7 +1238,7 @@ Content-Transfer-Encoding: 8bit
if os.path.exists(tarinfo._link_target): if os.path.exists(tarinfo._link_target):
os.link(tarinfo._link_target, targetpath) os.link(tarinfo._link_target, targetpath)
else: else:
@@ -2290,15 +2534,19 @@ class TarFile(object): @@ -2290,15 +2526,19 @@ def chown(self, tarinfo, targetpath, numeric_owner):
u = tarinfo.uid u = tarinfo.uid
if not numeric_owner: if not numeric_owner:
try: try:
@ -1269,7 +1260,7 @@ Content-Transfer-Encoding: 8bit
try: try:
if tarinfo.issym() and hasattr(os, "lchown"): if tarinfo.issym() and hasattr(os, "lchown"):
os.lchown(targetpath, u, g) os.lchown(targetpath, u, g)
@@ -2310,6 +2558,8 @@ class TarFile(object): @@ -2310,6 +2550,8 @@ def chown(self, tarinfo, targetpath, numeric_owner):
def chmod(self, tarinfo, targetpath): def chmod(self, tarinfo, targetpath):
"""Set file permissions of targetpath according to tarinfo. """Set file permissions of targetpath according to tarinfo.
""" """
@ -1278,7 +1269,7 @@ Content-Transfer-Encoding: 8bit
try: try:
os.chmod(targetpath, tarinfo.mode) os.chmod(targetpath, tarinfo.mode)
except OSError as e: except OSError as e:
@@ -2318,10 +2568,13 @@ class TarFile(object): @@ -2318,10 +2560,13 @@ def chmod(self, tarinfo, targetpath):
def utime(self, tarinfo, targetpath): def utime(self, tarinfo, targetpath):
"""Set modification time of targetpath according to tarinfo. """Set modification time of targetpath according to tarinfo.
""" """
@ -1293,7 +1284,7 @@ Content-Transfer-Encoding: 8bit
except OSError as e: except OSError as e:
raise ExtractError("could not change modification time") from e raise ExtractError("could not change modification time") from e
@@ -2397,13 +2650,26 @@ class TarFile(object): @@ -2397,13 +2642,26 @@ def _getmember(self, name, tarinfo=None, normalize=False):
members = self.getmembers() members = self.getmembers()
# Limit the member search list up to tarinfo. # Limit the member search list up to tarinfo.
@ -1321,7 +1312,7 @@ Content-Transfer-Encoding: 8bit
if normalize: if normalize:
member_name = os.path.normpath(member.name) member_name = os.path.normpath(member.name)
else: else:
@@ -2412,6 +2678,10 @@ class TarFile(object): @@ -2412,6 +2670,10 @@ def _getmember(self, name, tarinfo=None, normalize=False):
if name == member_name: if name == member_name:
return member return member
@ -1332,7 +1323,7 @@ Content-Transfer-Encoding: 8bit
def _load(self): def _load(self):
"""Read through the entire archive file and look for readable """Read through the entire archive file and look for readable
members. members.
@@ -2504,6 +2774,7 @@ class TarFile(object): @@ -2504,6 +2766,7 @@ def __exit__(self, type, value, traceback):
#-------------------- #--------------------
# exported functions # exported functions
#-------------------- #--------------------
@ -1340,7 +1331,7 @@ Content-Transfer-Encoding: 8bit
def is_tarfile(name): def is_tarfile(name):
"""Return True if name points to a tar archive that we """Return True if name points to a tar archive that we
are able to handle, else return False. are able to handle, else return False.
@@ -2530,6 +2801,10 @@ def main(): @@ -2530,6 +2793,10 @@ def main():
parser = argparse.ArgumentParser(description=description) parser = argparse.ArgumentParser(description=description)
parser.add_argument('-v', '--verbose', action='store_true', default=False, parser.add_argument('-v', '--verbose', action='store_true', default=False,
help='Verbose output') help='Verbose output')
@ -1351,7 +1342,7 @@ Content-Transfer-Encoding: 8bit
group = parser.add_mutually_exclusive_group(required=True) group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('-l', '--list', metavar='<tarfile>', group.add_argument('-l', '--list', metavar='<tarfile>',
help='Show listing of a tarfile') help='Show listing of a tarfile')
@@ -2541,8 +2816,12 @@ def main(): @@ -2541,8 +2808,12 @@ def main():
help='Create tarfile from sources') help='Create tarfile from sources')
group.add_argument('-t', '--test', metavar='<tarfile>', group.add_argument('-t', '--test', metavar='<tarfile>',
help='Test if a tarfile is valid') help='Test if a tarfile is valid')
@ -1364,7 +1355,7 @@ Content-Transfer-Encoding: 8bit
if args.test is not None: if args.test is not None:
src = args.test src = args.test
if is_tarfile(src): if is_tarfile(src):
@@ -2573,7 +2852,7 @@ def main(): @@ -2573,7 +2844,7 @@ def main():
if is_tarfile(src): if is_tarfile(src):
with TarFile.open(src, 'r:*') as tf: with TarFile.open(src, 'r:*') as tf:
@ -1373,9 +1364,11 @@ Content-Transfer-Encoding: 8bit
if args.verbose: if args.verbose:
if curdir == '.': if curdir == '.':
msg = '{!r} file is extracted.'.format(src) msg = '{!r} file is extracted.'.format(src)
diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py
index 0935b60d4c2..72fb3afcbef 100644
--- a/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py
+++ b/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py
@@ -32,6 +32,7 @@ except ImportError: @@ -32,6 +32,7 @@
from test import support from test import support
from test.support import os_helper from test.support import os_helper
from test.support.os_helper import TESTFN, FakePath from test.support.os_helper import TESTFN, FakePath
@ -1383,7 +1376,7 @@ Content-Transfer-Encoding: 8bit
TESTFN2 = TESTFN + "2" TESTFN2 = TESTFN + "2"
TESTFN_SRC = TESTFN + "_SRC" TESTFN_SRC = TESTFN + "_SRC"
@@ -1610,12 +1611,14 @@ class TestArchives(BaseTest, unittest.Te @@ -1610,12 +1611,14 @@ def test_register_archive_format(self):
### shutil.unpack_archive ### shutil.unpack_archive
@ -1403,7 +1396,7 @@ Content-Transfer-Encoding: 8bit
root_dir, base_dir = self._create_files() root_dir, base_dir = self._create_files()
expected = rlistdir(root_dir) expected = rlistdir(root_dir)
expected.remove('outer') expected.remove('outer')
@@ -1625,36 +1628,48 @@ class TestArchives(BaseTest, unittest.Te @@ -1625,36 +1628,47 @@ def check_unpack_archive_with_converter(self, format, converter):
# let's try to unpack it now # let's try to unpack it now
tmpdir2 = self.mkdtemp() tmpdir2 = self.mkdtemp()
@ -1428,8 +1421,7 @@ Content-Transfer-Encoding: 8bit
+ def check_unpack_tarball(self, format): + def check_unpack_tarball(self, format):
+ self.check_unpack_archive(format, filter='fully_trusted') + self.check_unpack_archive(format, filter='fully_trusted')
+ self.check_unpack_archive(format, filter='data') + self.check_unpack_archive(format, filter='data')
+ with warnings_helper.check_warnings( + with warnings_helper.check_no_warnings(self):
+ ('Python 3.14', DeprecationWarning)):
+ self.check_unpack_archive(format) + self.check_unpack_archive(format)
def test_unpack_archive_tar(self): def test_unpack_archive_tar(self):
@ -1460,14 +1452,12 @@ Content-Transfer-Encoding: 8bit
def test_unpack_registry(self): def test_unpack_registry(self):
diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py
index 89f5a561b4a..0d8d91b4d03 100644
--- a/Lib/test/test_tarfile.py --- a/Lib/test/test_tarfile.py
+++ b/Lib/test/test_tarfile.py +++ b/Lib/test/test_tarfile.py
@@ -2,9 +2,13 @@ import sys @@ -5,6 +5,10 @@
import os from contextlib import contextmanager
import io
from hashlib import sha256
-from contextlib import contextmanager
+from contextlib import contextmanager, ExitStack
from random import Random from random import Random
import pathlib import pathlib
+import shutil +import shutil
@ -1477,7 +1467,7 @@ Content-Transfer-Encoding: 8bit
import unittest import unittest
import unittest.mock import unittest.mock
@@ -13,6 +17,7 @@ import tarfile @@ -13,6 +17,7 @@
from test import support from test import support
from test.support import os_helper from test.support import os_helper
from test.support import script_helper from test.support import script_helper
@ -1485,7 +1475,7 @@ Content-Transfer-Encoding: 8bit
# Check for our compression modules. # Check for our compression modules.
try: try:
@@ -108,7 +113,7 @@ class UstarReadTest(ReadTest, unittest.T @@ -108,7 +113,7 @@ def test_fileobj_regular_file(self):
"regular file extraction failed") "regular file extraction failed")
def test_fileobj_readlines(self): def test_fileobj_readlines(self):
@ -1494,7 +1484,7 @@ Content-Transfer-Encoding: 8bit
tarinfo = self.tar.getmember("ustar/regtype") tarinfo = self.tar.getmember("ustar/regtype")
with open(os.path.join(TEMPDIR, "ustar/regtype"), "r") as fobj1: with open(os.path.join(TEMPDIR, "ustar/regtype"), "r") as fobj1:
lines1 = fobj1.readlines() lines1 = fobj1.readlines()
@@ -126,7 +131,7 @@ class UstarReadTest(ReadTest, unittest.T @@ -126,7 +131,7 @@ def test_fileobj_readlines(self):
"fileobj.readlines() failed") "fileobj.readlines() failed")
def test_fileobj_iter(self): def test_fileobj_iter(self):
@ -1503,7 +1493,7 @@ Content-Transfer-Encoding: 8bit
tarinfo = self.tar.getmember("ustar/regtype") tarinfo = self.tar.getmember("ustar/regtype")
with open(os.path.join(TEMPDIR, "ustar/regtype"), "r") as fobj1: with open(os.path.join(TEMPDIR, "ustar/regtype"), "r") as fobj1:
lines1 = fobj1.readlines() lines1 = fobj1.readlines()
@@ -136,7 +141,8 @@ class UstarReadTest(ReadTest, unittest.T @@ -136,7 +141,8 @@ def test_fileobj_iter(self):
"fileobj.__iter__() failed") "fileobj.__iter__() failed")
def test_fileobj_seek(self): def test_fileobj_seek(self):
@ -1513,7 +1503,7 @@ Content-Transfer-Encoding: 8bit
with open(os.path.join(TEMPDIR, "ustar/regtype"), "rb") as fobj: with open(os.path.join(TEMPDIR, "ustar/regtype"), "rb") as fobj:
data = fobj.read() data = fobj.read()
@@ -455,7 +461,7 @@ class CommonReadTest(ReadTest): @@ -455,7 +461,7 @@ def test_premature_end_of_archive(self):
t = tar.next() t = tar.next()
with self.assertRaisesRegex(tarfile.ReadError, "unexpected end of data"): with self.assertRaisesRegex(tarfile.ReadError, "unexpected end of data"):
@ -1522,7 +1512,7 @@ Content-Transfer-Encoding: 8bit
with self.assertRaisesRegex(tarfile.ReadError, "unexpected end of data"): with self.assertRaisesRegex(tarfile.ReadError, "unexpected end of data"):
tar.extractfile(t).read() tar.extractfile(t).read()
@@ -610,16 +616,16 @@ class MiscReadTestBase(CommonReadTest): @@ -610,16 +616,16 @@ def test_find_members(self):
def test_extract_hardlink(self): def test_extract_hardlink(self):
# Test hardlink extraction (e.g. bug #857297). # Test hardlink extraction (e.g. bug #857297).
with tarfile.open(tarname, errorlevel=1, encoding="iso8859-1") as tar: with tarfile.open(tarname, errorlevel=1, encoding="iso8859-1") as tar:
@ -1542,7 +1532,7 @@ Content-Transfer-Encoding: 8bit
self.addCleanup(os_helper.unlink, os.path.join(TEMPDIR, "ustar/symtype")) self.addCleanup(os_helper.unlink, os.path.join(TEMPDIR, "ustar/symtype"))
with open(os.path.join(TEMPDIR, "ustar/symtype"), "rb") as f: with open(os.path.join(TEMPDIR, "ustar/symtype"), "rb") as f:
data = f.read() data = f.read()
@@ -633,13 +639,14 @@ class MiscReadTestBase(CommonReadTest): @@ -633,13 +639,14 @@ def test_extractall(self):
os.mkdir(DIR) os.mkdir(DIR)
try: try:
directories = [t for t in tar if t.isdir()] directories = [t for t in tar if t.isdir()]
@ -1559,7 +1549,7 @@ Content-Transfer-Encoding: 8bit
def format_mtime(mtime): def format_mtime(mtime):
if isinstance(mtime, float): if isinstance(mtime, float):
return "{} ({})".format(mtime, mtime.hex()) return "{} ({})".format(mtime, mtime.hex())
@@ -662,7 +669,7 @@ class MiscReadTestBase(CommonReadTest): @@ -662,7 +669,7 @@ def test_extract_directory(self):
try: try:
with tarfile.open(tarname, encoding="iso8859-1") as tar: with tarfile.open(tarname, encoding="iso8859-1") as tar:
tarinfo = tar.getmember(dirtype) tarinfo = tar.getmember(dirtype)
@ -1568,7 +1558,7 @@ Content-Transfer-Encoding: 8bit
extracted = os.path.join(DIR, dirtype) extracted = os.path.join(DIR, dirtype)
self.assertEqual(os.path.getmtime(extracted), tarinfo.mtime) self.assertEqual(os.path.getmtime(extracted), tarinfo.mtime)
if sys.platform != "win32": if sys.platform != "win32":
@@ -675,7 +682,7 @@ class MiscReadTestBase(CommonReadTest): @@ -675,7 +682,7 @@ def test_extractall_pathlike_name(self):
with os_helper.temp_dir(DIR), \ with os_helper.temp_dir(DIR), \
tarfile.open(tarname, encoding="iso8859-1") as tar: tarfile.open(tarname, encoding="iso8859-1") as tar:
directories = [t for t in tar if t.isdir()] directories = [t for t in tar if t.isdir()]
@ -1577,7 +1567,7 @@ Content-Transfer-Encoding: 8bit
for tarinfo in directories: for tarinfo in directories:
path = DIR / tarinfo.name path = DIR / tarinfo.name
self.assertEqual(os.path.getmtime(path), tarinfo.mtime) self.assertEqual(os.path.getmtime(path), tarinfo.mtime)
@@ -686,7 +693,7 @@ class MiscReadTestBase(CommonReadTest): @@ -686,7 +693,7 @@ def test_extract_pathlike_name(self):
with os_helper.temp_dir(DIR), \ with os_helper.temp_dir(DIR), \
tarfile.open(tarname, encoding="iso8859-1") as tar: tarfile.open(tarname, encoding="iso8859-1") as tar:
tarinfo = tar.getmember(dirtype) tarinfo = tar.getmember(dirtype)
@ -1586,7 +1576,7 @@ Content-Transfer-Encoding: 8bit
extracted = DIR / dirtype extracted = DIR / dirtype
self.assertEqual(os.path.getmtime(extracted), tarinfo.mtime) self.assertEqual(os.path.getmtime(extracted), tarinfo.mtime)
@@ -1042,7 +1049,7 @@ class GNUReadTest(LongnameTest, ReadTest @@ -1042,7 +1049,7 @@ class GNUReadTest(LongnameTest, ReadTest, unittest.TestCase):
# an all platforms, and after that a test that will work only on # an all platforms, and after that a test that will work only on
# platforms/filesystems that prove to support sparse files. # platforms/filesystems that prove to support sparse files.
def _test_sparse_file(self, name): def _test_sparse_file(self, name):
@ -1595,7 +1585,7 @@ Content-Transfer-Encoding: 8bit
filename = os.path.join(TEMPDIR, name) filename = os.path.join(TEMPDIR, name)
with open(filename, "rb") as fobj: with open(filename, "rb") as fobj:
data = fobj.read() data = fobj.read()
@@ -1409,7 +1416,8 @@ class WriteTest(WriteTestBase, unittest. @@ -1409,7 +1416,8 @@ def test_extractall_symlinks(self):
with tarfile.open(temparchive, errorlevel=2) as tar: with tarfile.open(temparchive, errorlevel=2) as tar:
# this should not raise OSError: [Errno 17] File exists # this should not raise OSError: [Errno 17] File exists
try: try:
@ -1605,7 +1595,21 @@ Content-Transfer-Encoding: 8bit
except OSError: except OSError:
self.fail("extractall failed with symlinked files") self.fail("extractall failed with symlinked files")
finally: finally:
@@ -2441,6 +2449,15 @@ class CommandLineTest(unittest.TestCase) @@ -2406,7 +2414,12 @@ def test__all__(self):
'PAX_NUMBER_FIELDS', 'stn', 'nts', 'nti', 'itn', 'calc_chksums',
'copyfileobj', 'filemode', 'EmptyHeaderError',
'TruncatedHeaderError', 'EOFHeaderError', 'InvalidHeaderError',
- 'SubsequentHeaderError', 'ExFileObject', 'main'}
+ 'SubsequentHeaderError', 'ExFileObject', 'main',
+ "fully_trusted_filter", "data_filter",
+ "tar_filter", "FilterError", "AbsoluteLinkError",
+ "OutsideDestinationError", "SpecialFileError", "AbsolutePathError",
+ "LinkOutsideDestinationError",
+ }
support.check__all__(self, tarfile, not_exported=not_exported)
def test_useful_error_message_when_modules_missing(self):
@@ -2441,6 +2454,15 @@ def make_simple_tarfile(self, tar_name):
for tardata in files: for tardata in files:
tf.add(tardata, arcname=os.path.basename(tardata)) tf.add(tardata, arcname=os.path.basename(tardata))
@ -1621,7 +1625,7 @@ Content-Transfer-Encoding: 8bit
def test_bad_use(self): def test_bad_use(self):
rc, out, err = self.tarfilecmd_failure() rc, out, err = self.tarfilecmd_failure()
self.assertEqual(out, b'') self.assertEqual(out, b'')
@@ -2597,6 +2614,25 @@ class CommandLineTest(unittest.TestCase) @@ -2597,6 +2619,25 @@ def test_extract_command_verbose(self):
finally: finally:
os_helper.rmtree(tarextdir) os_helper.rmtree(tarextdir)
@ -1647,7 +1651,7 @@ Content-Transfer-Encoding: 8bit
def test_extract_command_different_directory(self): def test_extract_command_different_directory(self):
self.make_simple_tarfile(tmpname) self.make_simple_tarfile(tmpname)
try: try:
@@ -2680,7 +2716,7 @@ class LinkEmulationTest(ReadTest, unitte @@ -2680,7 +2721,7 @@ class LinkEmulationTest(ReadTest, unittest.TestCase):
# symbolic or hard links tarfile tries to extract these types of members # symbolic or hard links tarfile tries to extract these types of members
# as the regular files they point to. # as the regular files they point to.
def _test_link_extraction(self, name): def _test_link_extraction(self, name):
@ -1656,7 +1660,7 @@ Content-Transfer-Encoding: 8bit
with open(os.path.join(TEMPDIR, name), "rb") as f: with open(os.path.join(TEMPDIR, name), "rb") as f:
data = f.read() data = f.read()
self.assertEqual(sha256sum(data), sha256_regtype) self.assertEqual(sha256sum(data), sha256_regtype)
@@ -2812,8 +2848,10 @@ class NumericOwnerTest(unittest.TestCase @@ -2812,8 +2853,10 @@ def test_extract_with_numeric_owner(self, mock_geteuid, mock_chmod,
mock_chown): mock_chown):
with self._setup_test(mock_geteuid) as (tarfl, filename_1, _, with self._setup_test(mock_geteuid) as (tarfl, filename_1, _,
filename_2): filename_2):
@ -1669,7 +1673,7 @@ Content-Transfer-Encoding: 8bit
# convert to filesystem paths # convert to filesystem paths
f_filename_1 = os.path.join(TEMPDIR, filename_1) f_filename_1 = os.path.join(TEMPDIR, filename_1)
@@ -2831,7 +2869,8 @@ class NumericOwnerTest(unittest.TestCase @@ -2831,7 +2874,8 @@ def test_extractall_with_numeric_owner(self, mock_geteuid, mock_chmod,
mock_chown): mock_chown):
with self._setup_test(mock_geteuid) as (tarfl, filename_1, dirname_1, with self._setup_test(mock_geteuid) as (tarfl, filename_1, dirname_1,
filename_2): filename_2):
@ -1679,7 +1683,7 @@ Content-Transfer-Encoding: 8bit
# convert to filesystem paths # convert to filesystem paths
f_filename_1 = os.path.join(TEMPDIR, filename_1) f_filename_1 = os.path.join(TEMPDIR, filename_1)
@@ -2856,7 +2895,8 @@ class NumericOwnerTest(unittest.TestCase @@ -2856,7 +2900,8 @@ def test_extractall_with_numeric_owner(self, mock_geteuid, mock_chmod,
def test_extract_without_numeric_owner(self, mock_geteuid, mock_chmod, def test_extract_without_numeric_owner(self, mock_geteuid, mock_chmod,
mock_chown): mock_chown):
with self._setup_test(mock_geteuid) as (tarfl, filename_1, _, _): with self._setup_test(mock_geteuid) as (tarfl, filename_1, _, _):
@ -1689,7 +1693,7 @@ Content-Transfer-Encoding: 8bit
# convert to filesystem paths # convert to filesystem paths
f_filename_1 = os.path.join(TEMPDIR, filename_1) f_filename_1 = os.path.join(TEMPDIR, filename_1)
@@ -2870,6 +2910,873 @@ class NumericOwnerTest(unittest.TestCase @@ -2870,6 +2915,888 @@ def test_keyword_only(self, mock_geteuid):
tarfl.extract, filename_1, TEMPDIR, False, True) tarfl.extract, filename_1, TEMPDIR, False, True)
@ -1747,10 +1751,6 @@ Content-Transfer-Encoding: 8bit
+ tar = tarfile.open(tarname, mode='r', encoding="iso8859-1") + tar = tarfile.open(tarname, mode='r', encoding="iso8859-1")
+ cls.control_dir = pathlib.Path(TEMPDIR) / "extractall_ctrl" + cls.control_dir = pathlib.Path(TEMPDIR) / "extractall_ctrl"
+ tar.errorlevel = 0 + tar.errorlevel = 0
+ with ExitStack() as cm:
+ if cls.extraction_filter is None:
+ cm.enter_context(warnings.catch_warnings(
+ action="ignore", category=DeprecationWarning))
+ tar.extractall(cls.control_dir, filter=cls.extraction_filter) + tar.extractall(cls.control_dir, filter=cls.extraction_filter)
+ tar.close() + tar.close()
+ cls.control_paths = set( + cls.control_paths = set(
@ -2295,15 +2295,35 @@ Content-Transfer-Encoding: 8bit
+ arc.add('exec_group_other', mode='?rw-rwxrwx') + arc.add('exec_group_other', mode='?rw-rwxrwx')
+ arc.add('read_group_only', mode='?---r-----') + arc.add('read_group_only', mode='?---r-----')
+ arc.add('no_bits', mode='?---------') + arc.add('no_bits', mode='?---------')
+ arc.add('dir/', mode='?---rwsrwt', type=tarfile.DIRTYPE) + arc.add('dir/', mode='?---rwsrwt')
+
+ # On some systems, setting the sticky bit is a no-op.
+ # Check if that's the case.
+ tmp_filename = os.path.join(TEMPDIR, "tmp.file")
+ with open(tmp_filename, 'w'):
+ pass
+ os.chmod(tmp_filename, os.stat(tmp_filename).st_mode | stat.S_ISVTX)
+ have_sticky_files = (os.stat(tmp_filename).st_mode & stat.S_ISVTX)
+ os.unlink(tmp_filename)
+
+ os.mkdir(tmp_filename)
+ os.chmod(tmp_filename, os.stat(tmp_filename).st_mode | stat.S_ISVTX)
+ have_sticky_dirs = (os.stat(tmp_filename).st_mode & stat.S_ISVTX)
+ os.rmdir(tmp_filename)
+ +
+ with self.check_context(arc.open(), 'fully_trusted'): + with self.check_context(arc.open(), 'fully_trusted'):
+ if have_sticky_files:
+ self.expect_file('all_bits', mode='?rwsrwsrwt') + self.expect_file('all_bits', mode='?rwsrwsrwt')
+ else:
+ self.expect_file('all_bits', mode='?rwsrwsrwx')
+ self.expect_file('perm_bits', mode='?rwxrwxrwx') + self.expect_file('perm_bits', mode='?rwxrwxrwx')
+ self.expect_file('exec_group_other', mode='?rw-rwxrwx') + self.expect_file('exec_group_other', mode='?rw-rwxrwx')
+ self.expect_file('read_group_only', mode='?---r-----') + self.expect_file('read_group_only', mode='?---r-----')
+ self.expect_file('no_bits', mode='?---------') + self.expect_file('no_bits', mode='?---------')
+ self.expect_file('dir', type=tarfile.DIRTYPE, mode='?---rwsrwt') + if have_sticky_dirs:
+ self.expect_file('dir/', mode='?---rwsrwt')
+ else:
+ self.expect_file('dir/', mode='?---rwsrwx')
+ +
+ with self.check_context(arc.open(), 'tar'): + with self.check_context(arc.open(), 'tar'):
+ self.expect_file('all_bits', mode='?rwxr-xr-x') + self.expect_file('all_bits', mode='?rwxr-xr-x')
@ -2311,7 +2331,7 @@ Content-Transfer-Encoding: 8bit
+ self.expect_file('exec_group_other', mode='?rw-r-xr-x') + self.expect_file('exec_group_other', mode='?rw-r-xr-x')
+ self.expect_file('read_group_only', mode='?---r-----') + self.expect_file('read_group_only', mode='?---r-----')
+ self.expect_file('no_bits', mode='?---------') + self.expect_file('no_bits', mode='?---------')
+ self.expect_file('dir/', type=tarfile.DIRTYPE, mode='?---r-xr-x') + self.expect_file('dir/', mode='?---r-xr-x')
+ +
+ with self.check_context(arc.open(), 'data'): + with self.check_context(arc.open(), 'data'):
+ normal_dir_mode = stat.filemode(stat.S_IMODE( + normal_dir_mode = stat.filemode(stat.S_IMODE(
@ -2321,7 +2341,7 @@ Content-Transfer-Encoding: 8bit
+ self.expect_file('exec_group_other', mode='?rw-r--r--') + self.expect_file('exec_group_other', mode='?rw-r--r--')
+ self.expect_file('read_group_only', mode='?rw-r-----') + self.expect_file('read_group_only', mode='?rw-r-----')
+ self.expect_file('no_bits', mode='?rw-------') + self.expect_file('no_bits', mode='?rw-------')
+ self.expect_file('dir/', type=tarfile.DIRTYPE, mode=normal_dir_mode) + self.expect_file('dir/', mode=normal_dir_mode)
+ +
+ def test_pipe(self): + def test_pipe(self):
+ # Test handling of a special file + # Test handling of a special file
@ -2385,12 +2405,11 @@ Content-Transfer-Encoding: 8bit
+ self.assertIs(filtered.name, tarinfo.name) + self.assertIs(filtered.name, tarinfo.name)
+ self.assertIs(filtered.type, tarinfo.type) + self.assertIs(filtered.type, tarinfo.type)
+ +
+ def test_default_filter_warns(self): + def test_default_filter_warns_not(self):
+ """Ensure the default filter warns""" + """Ensure the default filter does not warn (like in 3.12)"""
+ with ArchiveMaker() as arc: + with ArchiveMaker() as arc:
+ arc.add('foo') + arc.add('foo')
+ with warnings_helper.check_warnings( + with warnings_helper.check_no_warnings(self):
+ ('Python 3.14', DeprecationWarning)):
+ with self.check_context(arc.open(), None): + with self.check_context(arc.open(), None):
+ self.expect_file('foo') + self.expect_file('foo')
+ +
@ -2563,6 +2582,9 @@ Content-Transfer-Encoding: 8bit
def setUpModule(): def setUpModule():
os_helper.unlink(TEMPDIR) os_helper.unlink(TEMPDIR)
os.makedirs(TEMPDIR) os.makedirs(TEMPDIR)
diff --git a/Misc/NEWS.d/next/Library/2023-03-23-15-24-38.gh-issue-102953.YR4KaK.rst b/Misc/NEWS.d/next/Library/2023-03-23-15-24-38.gh-issue-102953.YR4KaK.rst
new file mode 100644
index 00000000000..48a105a4a17
--- /dev/null --- /dev/null
+++ b/Misc/NEWS.d/next/Library/2023-03-23-15-24-38.gh-issue-102953.YR4KaK.rst +++ b/Misc/NEWS.d/next/Library/2023-03-23-15-24-38.gh-issue-102953.YR4KaK.rst
@@ -0,0 +1,4 @@ @@ -0,0 +1,4 @@