rpm/qemu
SHA256
1
0
Fork 0
forked from pool/qemu
Code Pull Requests Activity

Accepting request 990667 from home:dfaggioli:old_qemu

Browse Source 
- Substantial rework of the spec file: 
  * the 'make check' testsuite now runs in the %check section of
    the main package, not in a subpackage
  * switched from %setup to %autosetup
  * rearranged the content in order to minimize the use of %if,
    %ifarch, etc
- Properly fix bsc#1198038, CVE-2022-0216
* Patches added:
  scsi-lsi53c895a-really-fix-use-after-fre.patch
  tests-qtest-Move-the-fuzz-tests-to-x86-o.patch
- Make temp dir (for update_git.sh) configurable
- Added new subpackages (audio-dbus, ui-dbus)
- bsc#1199018 was never fixed in Factory's QEMU 6.2. It is
  now (since the patches are already in SeaBIOS 1.16.0)
- Some tests are having issues when run in OBS. They seem to be
  due to race conditions, triggered by resource constraints of
  OBS workers. Let's disable them for now, while looking for a fix
- Update to v7.0.0. For full release notese, see:
  * https://wiki.qemu.org/ChangeLog/7.0
  Be sure to also check the following pages:
  * https://qemu-project.gitlab.io/qemu/about/removed-features.html
  * https://qemu-project.gitlab.io/qemu/about/deprecated.html
  Some notable changes:
  * [ARM] The virt board has gained a new control knob to disable passing a RNG seed in the DTB (dtb-kaslr-seed)
  * [ARM] The AST2600 SoC now supports a dummy version of the i3c device
  * [ARM] The virt board can now run guests with KVM on hosts with restricted IPA ranges
  * [ARM] The virt board now supports virtio-mem-pci
  * [ARM] The virt board now supports specifying the guest CPU topology
  * [ARM] On the virt board, we now enable PAuth when using KVM or hvf and the host CPU supports it
  * [RISC-V] Add support for ratified 1.0 Vector extension
  * [RISC-V] Support for the Zve64f and Zve32f extensions
  * [RISC-V] Drop support for draft 0.7.1 Vector extension
  * [RISC-V] Support Zfhmin and Zfh extensions
  * [RISC-V] RISC-V KVM support
  * [RISC-V] Mark Hypervisor extension as non experimental
  * [RISC-V] Enable Hypervisor extension by default
  * [x86] Support for Intel AMX.
  * [PCI/PCIe] Q35: fix PCIe device becoming disabled after migration when ACPI based PCI hotplug is used (6b0969f1ec)
  * [PCI/PCIe] initial bits of SR/IOV support (250346169)
  * [PCI/PCIe] arm/virt: fixed PXB interrupt routing (e609301b45)
  * [PCI/PCIe] arm/virt: support for virtio-mem-pci (b1b87327a9)
  * [virtiofs] Fix for CVE-2022-0358 - behaviour with supplementary groups and SGID directories
  * [virtiofs] Improved security label support
  * [virtiofs] The virtiofsd in qemu is now starting to be deprecated; please start using and contributing to Rust virtiofsd
* Patches dropped:
  acpi-validate-hotplug-selector-on-access.patch
  block-backend-Retain-permissions-after-m.patch
  block-qdict-Fix-Werror-maybe-uninitializ.patch
  brotli-fix-actual-variable-array-paramet.patch
  display-qxl-render-fix-race-condition-in.patch
  doc-Add-the-SGX-numa-description.patch
  hw-i386-amd_iommu-Fix-maybe-uninitialize.patch
  hw-intc-exynos4210_gic-provide-more-room.patch
  hw-nvme-fix-CVE-2021-3929.patch
  hw-nvram-at24-return-0xff-if-1-byte-addr.patch
  iotest-065-explicit-compression-type.patch
  iotest-214-explicit-compression-type.patch
  iotest-302-use-img_info_log-helper.patch
  iotest-303-explicit-compression-type.patch
  iotest-39-use-_qcow2_dump_header.patch
  iotests-60-more-accurate-set-dirty-bit-i.patch
  iotests-bash-tests-filter-compression-ty.patch
  iotests-common.rc-introduce-_qcow2_dump_.patch
  iotests-declare-lack-of-support-for-comp.patch
  iotests-drop-qemu_img_verbose-helper.patch
  iotests-massive-use-_qcow2_dump_header.patch
  iotests-MRCE-Write-data-to-source.patch
  iotests.py-filter-out-successful-output-.patch
  iotests.py-img_info_log-rename-imgopts-a.patch
  iotests.py-implement-unsupported_imgopts.patch
  iotests.py-qemu_img-create-support-IMGOP.patch
  iotests.py-rewrite-default-luks-support-.patch
  iotests-specify-some-unsupported_imgopts.patch
  meson-build-all-modules-by-default.patch
  numa-Enable-numa-for-SGX-EPC-sections.patch
  numa-Support-SGX-numa-in-the-monitor-and.patch
  python-aqmp-add-__del__-method-to-legacy.patch
  python-aqmp-add-_session_guard.patch
  python-aqmp-add-SocketAddrT-to-package-r.patch
  python-aqmp-add-socket-bind-step-to-lega.patch
  python-aqmp-add-start_server-and-accept-.patch
  python-aqmp-copy-type-definitions-from-q.patch
  python-aqmp-drop-_bind_hack.patch
  python-aqmp-fix-docstring-typo.patch
  python-aqmp-Fix-negotiation-with-pre-oob.patch
  python-aqmp-fix-race-condition-in-legacy.patch
  Python-aqmp-fix-type-definitions-for-myp.patch
  python-aqmp-handle-asyncio.TimeoutError-.patch
  python-aqmp-refactor-_do_accept-into-two.patch
  python-aqmp-remove-_new_session-and-_est.patch
  python-aqmp-rename-accept-to-start_serve.patch
  python-aqmp-rename-AQMPError-to-QMPError.patch
  python-aqmp-split-_client_connected_cb-o.patch
  python-aqmp-squelch-pylint-warning-for-t.patch
  python-aqmp-stop-the-server-during-disco.patch
  python-introduce-qmp-shell-wrap-convenie.patch
  python-machine-raise-VMLaunchFailure-exc.patch
  python-move-qmp-shell-under-the-AQMP-pac.patch
  python-move-qmp-utilities-to-python-qemu.patch
  python-qmp-switch-qmp-shell-to-AQMP.patch
  python-support-recording-QMP-session-to-.patch
  python-upgrade-mypy-to-0.780.patch
  qcow2-simple-case-support-for-downgradin.patch
  qemu-binfmt-conf.sh-should-use-F-as-shor.patch
  tests-qemu-iotests-040-Skip-TestCommitWi.patch
  tests-qemu-iotests-Fix-051-for-binaries-.patch
  tests-qemu-iotests-testrunner-Quote-case.patch
  tools-virtiofsd-Add-rseq-syscall-to-the-.patch
  ui-cursor-fix-integer-overflow-in-cursor.patch
  vhost-vsock-detach-the-virqueue-element-.patch
  virtiofsd-Drop-membership-of-all-supplem.patch
  virtio-net-fix-map-leaking-on-error-duri.patch
  Disable-some-tests-that-have-problems-in.patch
* Patches added:
  intc-exynos4210_gic-replace-snprintf-wit.patch
  Revert-8dcb404bff6d9147765d7dd3e9c849337.patch
------------------------------------------------------------------
- Fix bsc#1197084
* Patches added:
  hostmem-default-the-amount-of-prealloc-t.patch
- Get rid of downstream patches breaking s390 modules. Replace
  them with the upstream proposed and Acked (but never committed)
  solution (bsc#1199015)
* Patches added:
  modules-generates-per-target-modinfo.patch
  modules-introduces-module_kconfig-direct.patch
* Patches dropped:
  Fix-the-module-building-problem-for-s390.patch
  modules-quick-fix-a-fundamental-error-in.patch
- backport patches for having coroutine work well when LTO is used
* Patches added:
  coroutine-ucontext-use-QEMU_DEFINE_STATI.patch
  coroutine-use-QEMU_DEFINE_STATIC_CO_TLS.patch
  coroutine-win32-use-QEMU_DEFINE_STATIC_C.patch
- seabios: drop patch that changes python in python2.
  Just go to python3 directly.
* Patches dropped:
  seabios-use-python2-explicitly-as-needed.patch

OBS-URL: https://build.opensuse.org/request/show/990667
OBS-URL: https://build.opensuse.org/package/show/Virtualization/qemu?expand=0&rev=726
This commit is contained in:
Dario Faggioli 2022-07-22 11:50:51 +00:00 committed by Git OBS Bridge
parent 6749a6e9ce
commit 5f7cc8fe5e
131 changed files with 5203 additions and 14038 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

BIN
DSDT.pcie Normal file
View File

Binary file not shown.

43
Disable-some-tests-that-have-problems-in.patch Normal file
View File

@ -0,0 +1,43 @@
From: Dario Faggioli <dfaggioli@suse.com>
Date: Thu, 21 Jul 2022 04:06:21 +0200
Subject: Disable some tests that have problems in OBS
Git-commit: 0000000000000000000000000000000000000000
We are disabling the following tests:
qemu-system-ppc64 / display-vga-test
They are failing due to some memory corruption errors. We believe that
this might be due to the combination of the compiler version and of LTO,
and will take up the investigation within the upstream community.
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
tests/qtest/meson.build | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
index e860b0edf275cd68cd24516cb1ca..ef1af9b109c1591e3bfda2cb4a56 100644
--- a/tests/qtest/meson.build
+++ b/tests/qtest/meson.build
@@ -35,6 +35,10 @@ qtests_pci = \
(config_all_devices.has_key('CONFIG_VGA') ? ['display-vga-test'] : []) + \
(config_all_devices.has_key('CONFIG_IVSHMEM_DEVICE') ? ['ivshmem-test'] : [])
+# display-vga-test currently fails in OBS, for ppc64
+qtests_pci_novga = \
+ (config_all_devices.has_key('CONFIG_IVSHMEM_DEVICE') ? ['ivshmem-test'] : [])
+
qtests_i386 = \
(slirp.found() ? ['pxe-test', 'test-netfilter'] : []) + \
(config_host.has_key('CONFIG_POSIX') ? ['test-filter-mirror'] : []) + \
@@ -171,7 +175,7 @@ qtests_ppc64 = \
(slirp.found() ? ['pxe-test'] : []) + \
(config_all_devices.has_key('CONFIG_USB_UHCI') ? ['usb-hcd-uhci-test'] : []) + \
(config_all_devices.has_key('CONFIG_USB_XHCI_NEC') ? ['usb-hcd-xhci-test'] : []) + \
- qtests_pci + ['migration-test', 'numa-test', 'cpu-plug-test', 'drive_del-test']
+ qtests_pci_novga + ['migration-test', 'numa-test', 'cpu-plug-test', 'drive_del-test']
qtests_sh4 = (config_all_devices.has_key('CONFIG_ISA_TESTDEV') ? ['endianness-test'] : [])
qtests_sh4eb = (config_all_devices.has_key('CONFIG_ISA_TESTDEV') ? ['endianness-test'] : [])

31
Fix-the-module-building-problem-for-s390.patch
View File

@ -1,31 +0,0 @@
From: Li Zhang <lizhang@suse.de>
Date: Thu, 24 Mar 2022 11:47:08 +0100
Subject: Fix the module building problem for s390
When building the testsuit for s390, it reports errors as the following:
hw-display-virtio-gpu.so: undefined symbol: virtio_vmstate_info
hw-display-virtio-gpu-gl.so: undefined symbol: virtio_gpu_ctrl_response
hw-display-virtio-gpu-pci.so: undefined symbol: virtio_pci_types_register
hw-display-virtio-gpu-pci-gl.so: undefined symbol: virtio_pci_types_register
Signed-off-by: Li Zhang <lizhang@suse.de>
---
util/module.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/util/module.c b/util/module.c
index 8370d4c6209eda4319342b202f6b..03896b7c469dedc5a80695aa7dee 100644
--- a/util/module.c
+++ b/util/module.c
@@ -125,6 +125,10 @@ bool s390x_blocklist(const char *name)
"hw-display-qxl",
"hw-display-virtio-vga",
"hw-display-virtio-vga-gl",
+ "hw-display-virtio-gpu",
+ "hw-display-virtio-gpu-gl",
+ "hw-display-virtio-gpu-pci",
+ "hw-display-virtio-gpu-pci-gl",
"hw-usb-host",
"hw-usb-redirect",
"hw-usb-smartcard"

10
Ignore-spurious-GCC-12-warning.patch
View File

@ -51,7 +51,7 @@ index 2ffefa8ceeeff9a1d0504ad71918..c9fb4329843a8fc34e596b506571 100644
// Template for an end-of-device path node.
//
diff --git a/roms/edk2/BaseTools/Source/C/GenFfs/GenFfs.c b/roms/edk2/BaseTools/Source/C/GenFfs/GenFfs.c
index fcb911f4fc34e560fd06001f91dd..589ffe94982ccfa3651216d0571f 100644
index 949025c333251bc5776159a6c535..be55a529743494677f8515906c6c 100644
--- a/roms/edk2/BaseTools/Source/C/GenFfs/GenFfs.c
+++ b/roms/edk2/BaseTools/Source/C/GenFfs/GenFfs.c
@@ -36,6 +36,10 @@ SPDX-License-Identifier: BSD-2-Clause-Patent
@ -81,12 +81,12 @@ index d54a4f9e0a7d67b7c8494ab37011..fb40ad36584c1cd53cc7ca4a9d4f 100644
NULL, // 0x00 - reserved
"EFI_SECTION_COMPRESSION", // 0x01
diff --git a/roms/edk2/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c b/roms/edk2/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
index e281716fee1bfb5eb236c9e38dbd..773114584add6e1f0f6e966e26e5 100644
index 4e9b499f8d80dc4d6bc13515e794..ab9b7cc34f69bceb74c454ce5032 100644
--- a/roms/edk2/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
+++ b/roms/edk2/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
@@ -38,6 +38,10 @@ static unsigned g_STAT_OFFSET = 0;
#define kNumBitPriceShiftBits 4
#define kBitPrice (1 << kNumBitPriceShiftBits)
@@ -40,6 +40,10 @@ static unsigned g_STAT_OFFSET = 0;
#define REP_LEN_COUNT 64
+#if __GNUC__ >= 12
+# pragma GCC diagnostic ignored "-Wdangling-pointer"

2
Makefile-Don-t-check-pc-bios-as-pre-requ.patch
View File

@ -13,7 +13,7 @@ Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Makefile b/Makefile
index 74c5b46d38b501f9dd25f447ed08..87ae3ba6efe74e6c48d2a6db970f 100644
index e5fd1ebdf6199c4cb15780e90236..e5881a951fc05d479b032c0b56d8 100644
--- a/Makefile
+++ b/Makefile
@@ -87,7 +87,7 @@ x := $(shell rm -rf meson-private meson-info meson-logs)

16
Makefile-fix-build-with-binutils-2.38.patch
View File

@ -29,20 +29,20 @@ Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/roms/opensbi/Makefile b/roms/opensbi/Makefile
index d6f097d30af78c0fba92eeeec523..a294f46fa8e16caec64d5a88c2a2 100644
index 062883e9695fb4447d2e61d7f282..fc1ea13e519a702e8b09d9006a2c 100644
--- a/roms/opensbi/Makefile
+++ b/roms/opensbi/Makefile
@@ -106,6 +106,9 @@ ifndef PLATFORM_RISCV_XLEN
endif
endif
@@ -153,6 +153,9 @@ OPENSBI_LD_PIE := $(shell $(CC) $(CLANG_TARGET) $(RELAX_FLAG) $(USE_LD_FLAG) -fP
# Check whether the compiler supports -m(no-)save-restore
CC_SUPPORT_SAVE_RESTORE := $(shell $(CC) $(CLANG_TARGET) $(RELAX_FLAG) -nostdlib -mno-save-restore -x c /dev/null -o /dev/null 2>&1 | grep "\-save\-restore" >/dev/null && echo n || echo y)
+# Check whether the assembler and the compiler support the Zicsr and Zifencei extensions
+CC_SUPPORT_ZICSR_ZIFENCEI := $(shell $(CC) $(CLANG_TARGET) $(RELAX_FLAG) -nostdlib -march=rv$(OPENSBI_CC_XLEN)imafd_zicsr_zifencei -x c /dev/null -o /dev/null 2>&1 | grep "zicsr\|zifencei" > /dev/null && echo n || echo y)
+
# Setup list of objects.mk files
ifdef PLATFORM
platform-object-mks=$(shell if [ -d $(platform_src_dir)/ ]; then find $(platform_src_dir) -iname "objects.mk" | sort -r; fi)
@@ -157,7 +160,11 @@ ifndef PLATFORM_RISCV_ABI
# Build Info:
# OPENSBI_BUILD_TIME_STAMP -- the compilation time stamp
# OPENSBI_BUILD_COMPILER_VERSION -- the compiler version info
@@ -223,7 +226,11 @@ ifndef PLATFORM_RISCV_ABI
endif
ifndef PLATFORM_RISCV_ISA
ifneq ($(PLATFORM_RISCV_TOOLCHAIN_DEFAULT), 1)

4
PPC-KVM-Disable-mmu-notifier-check.patch
View File

@ -14,10 +14,10 @@ Signed-off-by: Bruce Rogers <brogers@suse.com>
1 file changed, 2 insertions(+)
diff --git a/softmmu/physmem.c b/softmmu/physmem.c
index 3524c04c2a162b717c3975cc15da..f69d6b00467f8b53614171fa17a9 100644
index 4e1b27a20eccca027515aee6f78f..02aa7205a4992a678414111f625c 100644
--- a/softmmu/physmem.c
+++ b/softmmu/physmem.c
@@ -2064,11 +2064,13 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr,
@@ -2066,11 +2066,13 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr,
return NULL;
}

48
Python-aqmp-fix-type-definitions-for-myp.patch
View File

@ -1,48 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Mon, 10 Jan 2022 14:13:48 -0500
Subject: Python/aqmp: fix type definitions for mypy 0.920
Git-commit: 42d73f2894ea1855df5a25d58e0d9eac6023dcc3
0.920 (Released 2021-12-15) is not entirely happy with the
way that I was defining _FutureT:
qemu/aqmp/protocol.py:601: error: Item "object" of the upper bound
"Optional[Future[Any]]" of type variable "_FutureT" has no attribute
"done"
Update it with something a little mechanically simpler that works better
across a wider array of mypy versions.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20220110191349.1841027-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/protocol.py | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py
index 5190b33b13df24fc2ca4aed934ed..c4fbe35a0e41c589059ec4fa37a8 100644
--- a/python/qemu/aqmp/protocol.py
+++ b/python/qemu/aqmp/protocol.py
@@ -43,8 +43,8 @@ from .util import (
T = TypeVar('T')
+_U = TypeVar('_U')
_TaskFN = Callable[[], Awaitable[None]] # aka ``async def func() -> None``
-_FutureT = TypeVar('_FutureT', bound=Optional['asyncio.Future[Any]'])
class Runstate(Enum):
@@ -591,7 +591,8 @@ class AsyncProtocol(Generic[T]):
"""
Fully reset this object to a clean state and return to `IDLE`.
"""
- def _paranoid_task_erase(task: _FutureT) -> Optional[_FutureT]:
+ def _paranoid_task_erase(task: Optional['asyncio.Future[_U]']
+ ) -> Optional['asyncio.Future[_U]']:
# Help to erase a task, ENSURING it is fully quiesced first.
assert (task is None) or task.done()
return None if (task and task.done()) else task

6
Raise-soft-address-space-limit-to-hard-l.patch
View File

@ -16,10 +16,10 @@ Signed-off-by: Bruce Rogers <brogers@suse.com>
1 file changed, 12 insertions(+)
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 620a1f1367e2b033bfec541619d3..527e1d91859b19351d397ef64930 100644
index 6f646531a0aeb6ff2dd3fc8a5cd8..f0f2f190a4a84af95ccaac353bb1 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -40,6 +40,7 @@
@@ -41,6 +41,7 @@
#include "sysemu/reset.h"
#include "sysemu/runstate.h"
#include "sysemu/runstate-action.h"
@ -27,7 +27,7 @@ index 620a1f1367e2b033bfec541619d3..527e1d91859b19351d397ef64930 100644
#include "sysemu/seccomp.h"
#include "sysemu/tcg.h"
#include "sysemu/xen.h"
@@ -2772,6 +2773,17 @@ void qemu_init(int argc, char **argv, char **envp)
@@ -2778,6 +2779,17 @@ void qemu_init(int argc, char **argv, char **envp)
MachineClass *machine_class;
bool userconfig = true;
FILE *vmstate_dump_file = NULL;

55
Revert-8dcb404bff6d9147765d7dd3e9c849337.patch Normal file
View File

@ -0,0 +1,55 @@
From: Dario Faggioli <dfaggioli@suse.com>
Date: Tue, 24 May 2022 20:52:17 +0200
Subject: Revert 8dcb404bff6d9147765d7dd3e9c8493372186420
Git-commit: 0000000000000000000000000000000000000000
Revert commit "tests/qtest: enable more vhost-user tests by default"
(8dcb404bff6d9), as it causes prooblem when building with GCC 12 and LTO
enabled.
This should be considered temporary, until the actual reason why the
code of the tests that are added in that commit breaks.
It has been reported upstream, and will be (hopefully) solved there:
https://lore.kernel.org/qemu-devel/1d3bbff9e92e7c8a24db9e140dcf3f428c2df103.camel@suse.com/
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
tests/qtest/vhost-user-test.c | 21 ++++++++++++---------
1 file changed, 12 insertions(+), 9 deletions(-)
diff --git a/tests/qtest/vhost-user-test.c b/tests/qtest/vhost-user-test.c
index ee30f54796485e1fb2cab1fe7641..6e79935c47e6f7fab9741cdf9e50 100644
--- a/tests/qtest/vhost-user-test.c
+++ b/tests/qtest/vhost-user-test.c
@@ -995,17 +995,20 @@ static void register_vhost_user_test(void)
"virtio-net",
test_migrate, &opts);
- opts.before = vhost_user_test_setup_reconnect;
- qos_add_test("vhost-user/reconnect", "virtio-net",
- test_reconnect, &opts);
+ /* keeps failing on build-system since Aug 15 2017 */
+ if (getenv("QTEST_VHOST_USER_FIXME")) {
+ opts.before = vhost_user_test_setup_reconnect;
+ qos_add_test("vhost-user/reconnect", "virtio-net",
+ test_reconnect, &opts);
- opts.before = vhost_user_test_setup_connect_fail;
- qos_add_test("vhost-user/connect-fail", "virtio-net",
- test_vhost_user_started, &opts);
+ opts.before = vhost_user_test_setup_connect_fail;
+ qos_add_test("vhost-user/connect-fail", "virtio-net",
+ test_vhost_user_started, &opts);
- opts.before = vhost_user_test_setup_flags_mismatch;
- qos_add_test("vhost-user/flags-mismatch", "virtio-net",
- test_vhost_user_started, &opts);
+ opts.before = vhost_user_test_setup_flags_mismatch;
+ qos_add_test("vhost-user/flags-mismatch", "virtio-net",
+ test_vhost_user_started, &opts);
+ }
opts.before = vhost_user_test_setup_multiqueue;
opts.edge.extra_device_opts = "mq=on";

6
Revert-qht-constify-qht_statistics_init.patch
View File

@ -27,10 +27,10 @@ index 758c7ac6c89978ee762e9d946584..2e2d6bca932c83bea993850b60f8 100644
/**
* qht_statistics_destroy - Destroy a &struct qht_stats
diff --git a/util/qht.c b/util/qht.c
index 079605121b1452dac9937b0e6137..cfc2c60639af8f7905aae51c53bb 100644
index 065fc501f44c46dffac74134ee2c..0803c194d10b5cdafdc8b18ec083 100644
--- a/util/qht.c
+++ b/util/qht.c
@@ -901,9 +901,9 @@ bool qht_resize(struct qht *ht, size_t n_elems)
@@ -902,9 +902,9 @@ bool qht_resize(struct qht *ht, size_t n_elems)
}
/* pass @stats to qht_statistics_destroy() when done */
@ -42,7 +42,7 @@ index 079605121b1452dac9937b0e6137..cfc2c60639af8f7905aae51c53bb 100644
int i;
map = qatomic_rcu_read(&ht->map);
@@ -920,8 +920,8 @@ void qht_statistics_init(const struct qht *ht, struct qht_stats *stats)
@@ -921,8 +921,8 @@ void qht_statistics_init(const struct qht *ht, struct qht_stats *stats)
stats->head_buckets = map->n_buckets;
for (i = 0; i < map->n_buckets; i++) {

2
Revert-roms-efirom-tests-uefi-test-tools.patch
View File

@ -15,7 +15,7 @@ Signed-off-by: Bruce Rogers <brogers@suse.com>
2 files changed, 3 deletions(-)
diff --git a/roms/Makefile b/roms/Makefile
index 66d06f5831303c3d41e943290389..6ea8edd9fcf6bb0cdc1f1602f241 100644
index 3d53bd528666b594ab3d6b631450..c7c6e973db59572e8b8bee74e969 100644
--- a/roms/Makefile
+++ b/roms/Makefile
@@ -151,8 +151,6 @@ build-efi-roms: build-pxe-roms

8
XXX-dont-dump-core-on-sigabort.patch
View File

@ -8,11 +8,11 @@ Signed-off-by: Bruce Rogers <brogers@suse.com>
1 file changed, 6 insertions(+)
diff --git a/linux-user/signal.c b/linux-user/signal.c
index 6d5e5b698cc827416b4fdf78aac3..41ec78926582729c91f486f1fb15 100644
index 092e70b80c6ffb83fc52ff6d6a0a..b918084cc769c9bc1d15fbfad792 100644
--- a/linux-user/signal.c
+++ b/linux-user/signal.c
@@ -734,6 +734,10 @@ static void QEMU_NORETURN dump_core_and_abort(int target_sig)
trace_user_force_sig(env, target_sig, host_sig);
@@ -737,6 +737,10 @@ static void QEMU_NORETURN dump_core_and_abort(int target_sig)
trace_user_dump_core_and_abort(env, target_sig, host_sig);
gdb_signalled(env, target_sig);
+ if (target_sig == 6) {
@ -22,7 +22,7 @@ index 6d5e5b698cc827416b4fdf78aac3..41ec78926582729c91f486f1fb15 100644
/* dump core if supported by target binary format */
if (core_dump_signal(target_sig) && (ts->bprm->core_dump != NULL)) {
stop_all_tasks();
@@ -751,6 +755,8 @@ static void QEMU_NORETURN dump_core_and_abort(int target_sig)
@@ -754,6 +758,8 @@ static void QEMU_NORETURN dump_core_and_abort(int target_sig)
target_sig, strsignal(host_sig), "core dumped" );
}

1
_multibuild
View File

@ -1,4 +1,3 @@
<multibuild>
<package>linux-user</package>
<package>testsuite</package>
</multibuild>

41
acpi-validate-hotplug-selector-on-access.patch
View File

@ -1,41 +0,0 @@
From: "Michael S. Tsirkin" <mst@redhat.com>
Date: Tue, 21 Dec 2021 09:45:44 -0500
Subject: acpi: validate hotplug selector on access
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 9bd6565ccee68f72d5012e24646e12a1c662827e
References: bsc#1194063
When bus is looked up on a pci write, we didn't
validate that the lookup succeeded.
Fuzzers thus can trigger QEMU crash by dereferencing the NULL
bus pointer.
Fixes: b32bd763a1 ("pci: introduce acpi-index property for PCI device")
Fixes: CVE-2021-4158
Cc: "Igor Mammedov" <imammedo@redhat.com>
Fixes: https://gitlab.com/qemu-project/qemu/-/issues/770
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Ani Sinha <ani@anisinha.ca>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
hw/acpi/pcihp.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/hw/acpi/pcihp.c b/hw/acpi/pcihp.c
index 30405b5113d7fb5a259a9d9cbd76..a5e182dd3a372fee2805923a314c 100644
--- a/hw/acpi/pcihp.c
+++ b/hw/acpi/pcihp.c
@@ -491,6 +491,9 @@ static void pci_write(void *opaque, hwaddr addr, uint64_t data,
}
bus = acpi_pcihp_find_hotplug_bus(s, s->hotplug_select);
+ if (!bus) {
+ break;
+ }
QTAILQ_FOREACH_SAFE(kid, &bus->qbus.children, sibling, next) {
Object *o = OBJECT(kid->child);
PCIDevice *dev = PCI_DEVICE(o);

4
acpi_piix4-Fix-migration-from-SLE11-SP2.patch
View File

@ -16,10 +16,10 @@ Signed-off-by: Andreas Färber <afaerber@suse.de>
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
index f0b5fac44a14279ac3c66834bfb1..42c1a7a816cbdb22b16876dc6f8c 100644
index fe5625d07a289800831170f7e41f..dd005ff2fc097f72947a62d91b96 100644
--- a/hw/acpi/piix4.c
+++ b/hw/acpi/piix4.c
@@ -278,7 +278,7 @@ static bool piix4_vmstate_need_smbus(void *opaque, int version_id)
@@ -287,7 +287,7 @@ static bool vmstate_test_migrate_acpi_index(void *opaque, int version_id)
static const VMStateDescription vmstate_acpi = {
.name = "piix4_pm",
.version_id = 3,

72
block-backend-Retain-permissions-after-m.patch
View File

@ -1,72 +0,0 @@
From: Hanna Reitz <hreitz@redhat.com>
Date: Thu, 25 Nov 2021 14:53:16 +0100
Subject: block-backend: Retain permissions after migration
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 492a119610129f65217580790971fa038e5492d3
After migration, the permissions the guest device wants to impose on its
BlockBackend are stored in blk->perm and blk->shared_perm. In
blk_root_activate(), we take our permissions, but keep all shared
permissions open by calling `blk_set_perm(blk->perm, BLK_PERM_ALL)`.
Only afterwards (immediately or later, depending on the runstate) do we
restrict the shared permissions by calling
`blk_set_perm(blk->perm, blk->shared_perm)`. Unfortunately, our first
call with shared_perm=BLK_PERM_ALL has overwritten blk->shared_perm to
be BLK_PERM_ALL, so this is a no-op and the set of shared permissions is
not restricted.
Fix this bug by saving the set of shared permissions before invoking
blk_set_perm() with BLK_PERM_ALL and restoring it afterwards.
Fixes: 5f7772c4d0cf32f4e779fcd5a69ae4dae24aeebf
("block-backend: Defer shared_perm tightening migration
completion")
Reported-by: Peng Liang <liangpeng10@huawei.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211125135317.186576-2-hreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Peng Liang <liangpeng10@huawei.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
block/block-backend.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/block/block-backend.c b/block/block-backend.c
index 12ef80ea170c04a15500b42bc5e7..41e388fe1f30af8f3cf6a872b37a 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -190,6 +190,7 @@ static void blk_root_activate(BdrvChild *child, Error **errp)
{
BlockBackend *blk = child->opaque;
Error *local_err = NULL;
+ uint64_t saved_shared_perm;
if (!blk->disable_perm) {
return;
@@ -197,12 +198,22 @@ static void blk_root_activate(BdrvChild *child, Error **errp)
blk->disable_perm = false;
+ /*
+ * blk->shared_perm contains the permissions we want to share once
+ * migration is really completely done. For now, we need to share
+ * all; but we also need to retain blk->shared_perm, which is
+ * overwritten by a successful blk_set_perm() call. Save it and
+ * restore it below.
+ */
+ saved_shared_perm = blk->shared_perm;
+
blk_set_perm(blk, blk->perm, BLK_PERM_ALL, &local_err);
if (local_err) {
error_propagate(errp, local_err);
blk->disable_perm = true;
return;
}
+ blk->shared_perm = saved_shared_perm;
if (runstate_check(RUN_STATE_INMIGRATE)) {
/* Activation can happen when migration process is still active, for

64
block-qdict-Fix-Werror-maybe-uninitializ.patch
View File

@ -1,64 +0,0 @@
From: Murilo Opsfelder Araujo <muriloo@linux.ibm.com>
Date: Fri, 11 Mar 2022 19:16:34 -0300
Subject: block-qdict: Fix -Werror=maybe-uninitialized build failure
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 79854b95446396c8e7f397800c5d86c3d9a7540d
References: bsc#1199625
Building QEMU on Fedora 37 (Rawhide Prerelease) ppc64le failed with the
following error:
$ ../configure --prefix=/usr/local/qemu-disabletcg --target-list=ppc-softmmu,ppc64-softmmu --disable-tcg --disable-linux-user
...
$ make -j$(nproc)
...
In file included from /root/qemu/include/qapi/qmp/qdict.h:16,
from /root/qemu/include/block/qdict.h:13,
from ../qobject/block-qdict.c:11:
/root/qemu/include/qapi/qmp/qobject.h: In function ‘qdict_array_split’:
/root/qemu/include/qapi/qmp/qobject.h:49:17: error: ‘subqdict’ may be used uninitialized [-Werror=maybe-uninitialized]
49 | typeof(obj) _obj = (obj); \
| ^~~~
../qobject/block-qdict.c:227:16: note: ‘subqdict’ declared here
227 | QDict *subqdict;
| ^~~~~~~~
cc1: all warnings being treated as errors
Fix build failure by expanding the ternary operation.
Tested with `make check-unit` (the check-block-qdict test passed).
Signed-off-by: Murilo Opsfelder Araujo <muriloo@linux.ibm.com>
Cc: Kevin Wolf <kwolf@redhat.com>
Cc: Hanna Reitz <hreitz@redhat.com>
Cc: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220311221634.58288-1-muriloo@linux.ibm.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
qobject/block-qdict.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c
index 1487cc5dd8b1cfbd50b9f354bcfa..4a83bda2c3be54994ae6a704b20b 100644
--- a/qobject/block-qdict.c
+++ b/qobject/block-qdict.c
@@ -251,12 +251,12 @@ void qdict_array_split(QDict *src, QList **dst)
if (is_subqdict) {
qdict_extract_subqdict(src, &subqdict, prefix);
assert(qdict_size(subqdict) > 0);
+ qlist_append_obj(*dst, QOBJECT(subqdict));
} else {
qobject_ref(subqobj);
qdict_del(src, indexstr);
+ qlist_append_obj(*dst, subqobj);
}
-
- qlist_append_obj(*dst, subqobj ?: QOBJECT(subqdict));
}
}

49
brotli-fix-actual-variable-array-paramet.patch
View File

@ -1,49 +0,0 @@
From: Bruce Rogers <brogers@suse.com>
Date: Fri, 5 Mar 2021 13:25:44 -0700
Subject: brotli: fix actual variable-array parameters to match declaration
References: boo#1181922
GCC 11 complains about the mismatch between the variable-array
parameters not being defined as such in the actual function definition.
Make them match.
Signed-off-by: Bruce Rogers <brogers@suse.com>
---
c/dec/decode.c | 6 ++++--
c/enc/encode.c | 5 +++--
2 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/roms/edk2/BaseTools/Source/C/BrotliCompress/brotli/c/dec/decode.c b/roms/edk2/BaseTools/Source/C/BrotliCompress/brotli/c/dec/decode.c
index 114c5057d7d91f5a634167e00d84..bb6f1ab6cc42f54720fd07f665e3 100644
--- a/roms/edk2/BaseTools/Source/C/BrotliCompress/brotli/c/dec/decode.c
+++ b/roms/edk2/BaseTools/Source/C/BrotliCompress/brotli/c/dec/decode.c
@@ -2030,8 +2030,10 @@ static BROTLI_NOINLINE BrotliDecoderErrorCode SafeProcessCommands(
}
BrotliDecoderResult BrotliDecoderDecompress(
- size_t encoded_size, const uint8_t* encoded_buffer, size_t* decoded_size,
- uint8_t* decoded_buffer) {
+ size_t encoded_size,
+ const uint8_t encoded_buffer[BROTLI_ARRAY_PARAM(encoded_size)],
+ size_t* decoded_size,
+ uint8_t decoded_buffer[BROTLI_ARRAY_PARAM(*decoded_size)]) {
BrotliDecoderState s;
BrotliDecoderResult result;
size_t total_out = 0;
diff --git a/roms/edk2/BaseTools/Source/C/BrotliCompress/brotli/c/enc/encode.c b/roms/edk2/BaseTools/Source/C/BrotliCompress/brotli/c/enc/encode.c
index 68548ef55af8036d6f0fe720e721..ab0a4906060c98461102d45ab61d 100644
--- a/roms/edk2/BaseTools/Source/C/BrotliCompress/brotli/c/enc/encode.c
+++ b/roms/edk2/BaseTools/Source/C/BrotliCompress/brotli/c/enc/encode.c
@@ -1470,8 +1470,9 @@ static size_t MakeUncompressedStream(
BROTLI_BOOL BrotliEncoderCompress(
int quality, int lgwin, BrotliEncoderMode mode, size_t input_size,
- const uint8_t* input_buffer, size_t* encoded_size,
- uint8_t* encoded_buffer) {
+ const uint8_t input_buffer[BROTLI_ARRAY_PARAM(input_size)],
+ size_t* encoded_size,
+ uint8_t encoded_buffer[BROTLI_ARRAY_PARAM(*encoded_size)]) {
BrotliEncoderState* s;
size_t out_size = *encoded_size;
const uint8_t* input_start = input_buffer;

4
bundles.tar.xz
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:caff72ba0d91116cb012ed88bd6f4cce2ee7015889cb1d1502abfdfd8d73dbd7
size 146284
oid sha256:a91840fe3ca2e630233b1a545007fdf03430eebc54d9eeae015f17e32f036c90
size 68168

21
config.sh
View File

@ -18,12 +18,12 @@ UPSTREAM_GIT_REPO=https://gitlab.com/qemu-project/qemu.git
# The following specifies the upstream tag or commit upon which our patchqueue
# gets rebased. The special value LATEST may be used to "automatically" track
# the upstream development tree in the master branch
GIT_UPSTREAM_COMMIT_ISH=v6.2.0
GIT_UPSTREAM_COMMIT_ISH=v7.0.0
# WARNING: If transitioning from using LATEST to not, MANUALLY re-set the
# tarball present. If transitioning TO LATEST, make sure that
# NEXT_RELEASE_IS_MAJOR is set correctly
# This is used to choose the version number when LATEST processing is active
NEXT_RELEASE_IS_MAJOR=1
NEXT_RELEASE_IS_MAJOR=0
# Unfortunately, SeaBIOS doesn't always follow an "always increasing" version
# model, so there may be times we should overide the automated version setting.
@ -38,6 +38,11 @@ PATCH_RANGE=1000
# For compatibility with old packages, we include this option
OVERRIDE_FIVE_DIGIT_NUMBERING=0
# Path to be used for temporary files, directories, repositories, etc.
# Default is /dev/shm. An alternative could be /tmp (e.g., when building
# in containers, or whatever).
TMPDIR=/dev/shm
# This array tracks all git submodule paths within the superproject (1st entry)
PATCH_PATH_MAP=(
""
@ -63,6 +68,7 @@ PATCH_PATH_MAP=(
"meson/"
"tests/fp/berkeley-softfloat-3/"
"tests/fp/berkeley-testfloat-3/"
"tests/lcitool/libvirt-ci"
"roms/edk2/ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3/"
"roms/edk2/CryptoPkg/Library/OpensslLib/openssl/boringssl/"
"roms/edk2/CryptoPkg/Library/OpensslLib/openssl/krb5/"
@ -72,10 +78,7 @@ PATCH_PATH_MAP=(
"roms/edk2/MdeModulePkg/Universal/RegularExpressionDxe/oniguruma/"
"roms/edk2/UnitTestFrameworkPkg/Library/CmockaLib/cmocka/"
"roms/vbootrom/"
"roms/edk2/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli/research/esaxx/"
"roms/edk2/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli/research/libdivsufsort/"
"roms/edk2/MdeModulePkg/Library/BrotliCompress/brotli/research/esaxx/"
"roms/edk2/MdeModulePkg/Library/BrotliCompress/brotli/research/libdivsufsort/"
"roms/edk2/RedfishPkg/Library/JsonLib/jansson"
)
# (order and count must correspond to PATCH_PATH_MAP)
@ -103,6 +106,7 @@ LOCAL_REPO_MAP=(
~/git/qemu-meson
~/git/qemu-tests-berkeley-softfloat-3
~/git/qemu-tests-berkeley-testfloat-3
~/git/qemu-tests-lcitool-libvirt-ci
~/git/qemu-edk2-berkeley-softfloat-3
~/git/qemu-edk2-openssl-boringssl
~/git/qemu-edk2-openssl-krb5
@ -112,8 +116,5 @@ LOCAL_REPO_MAP=(
~/git/qemu-edk2-oniguruma
~/git/qemu-edk2-cmocka
~/git/qemu-vbootrom
~/git/qemu-edk2-BrotliCustomDecompressLib-brotli-research-esaxx
~/git/qemu-edk2-BrotliCustomDecompressLib-brotli-research-libdivsufsort
~/git/qemu-edk2-BrotliCompress-brotli-research-esaxx
~/git/qemu-edk2-BrotliCompress-brotli-research-libdivsufsort
~/git/qemu-edk2-jansson
)

4
configure-only-populate-roms-if-softmmu.patch
View File

@ -12,10 +12,10 @@ Signed-off-by: Bruce Rogers <brogers@suse.com>
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/configure b/configure
index 48c21775f3a90c91631d90bd6e3e..e53fc9b91a69870e354ba2f27475 100755
index 7c08c18358becf49779c876b0f3d..a35b63b2106900e6f7ff46d2b8da 100755
--- a/configure
+++ b/configure
@@ -3373,7 +3373,7 @@ fi
@@ -2725,7 +2725,7 @@ fi
# Only build s390-ccw bios if we're on s390x and the compiler has -march=z900
# or -march=z10 (which is the lowest architecture level that Clang supports)

4
configure-remove-pkgversion-from-CONFIG_.patch
View File

@ -18,10 +18,10 @@ Signed-off-by: Bruce Rogers <brogers@suse.com>
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/configure b/configure
index e53fc9b91a69870e354ba2f27475..5312dc66dd293857ef916f3dde44 100755
index a35b63b2106900e6f7ff46d2b8da..b031106ff395bf5092348780dbb6 100755
--- a/configure
+++ b/configure
@@ -3509,7 +3509,7 @@ echo "TARGET_DIRS=$target_list" >> $config_host_mak
@@ -2823,7 +2823,7 @@ echo "TARGET_DIRS=$target_list" >> $config_host_mak
if test "$modules" = "yes"; then
# $shacmd can generate a hash started with digit, which the compiler doesn't
# like as an symbol. So prefix it with an underscore

122
coroutine-ucontext-use-QEMU_DEFINE_STATI.patch Normal file
View File

@ -0,0 +1,122 @@
From: Stefan Hajnoczi <stefanha@redhat.com>
Date: Mon, 7 Mar 2022 15:38:51 +0000
Subject: coroutine-ucontext: use QEMU_DEFINE_STATIC_CO_TLS()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 34145a307d849d0b6734d0222a7aa0bb9eef7407
Thread-Local Storage variables cannot be used directly from coroutine
code because the compiler may optimize TLS variable accesses across
qemu_coroutine_yield() calls. When the coroutine is re-entered from
another thread the TLS variables from the old thread must no longer be
used.
Use QEMU_DEFINE_STATIC_CO_TLS() for the current and leader variables.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220307153853.602859-2-stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
[DF: see: https://bugzilla.redhat.com/show_bug.cgi?id=1952483]
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
util/coroutine-ucontext.c | 38 ++++++++++++++++++++++++--------------
1 file changed, 24 insertions(+), 14 deletions(-)
diff --git a/util/coroutine-ucontext.c b/util/coroutine-ucontext.c
index 904b375192cab6d24c2bdea46d8c..127d5a13c8e77cae82cbb7cb5129 100644
--- a/util/coroutine-ucontext.c
+++ b/util/coroutine-ucontext.c
@@ -25,6 +25,7 @@
#include "qemu/osdep.h"
#include <ucontext.h>
#include "qemu/coroutine_int.h"
+#include "qemu/coroutine-tls.h"
#ifdef CONFIG_VALGRIND_H
#include <valgrind/valgrind.h>
@@ -66,8 +67,8 @@ typedef struct {
/**
* Per-thread coroutine bookkeeping
*/
-static __thread CoroutineUContext leader;
-static __thread Coroutine *current;
+QEMU_DEFINE_STATIC_CO_TLS(Coroutine *, current);
+QEMU_DEFINE_STATIC_CO_TLS(CoroutineUContext, leader);
/*
* va_args to makecontext() must be type 'int', so passing
@@ -97,14 +98,15 @@ static inline __attribute__((always_inline))
void finish_switch_fiber(void *fake_stack_save)
{
#ifdef CONFIG_ASAN
+ CoroutineUContext *leaderp = get_ptr_leader();
const void *bottom_old;
size_t size_old;
__sanitizer_finish_switch_fiber(fake_stack_save, &bottom_old, &size_old);
- if (!leader.stack) {
- leader.stack = (void *)bottom_old;
- leader.stack_size = size_old;
+ if (!leaderp->stack) {
+ leaderp->stack = (void *)bottom_old;
+ leaderp->stack_size = size_old;
}
#endif
#ifdef CONFIG_TSAN
@@ -161,8 +163,10 @@ static void coroutine_trampoline(int i0, int i1)
/* Initialize longjmp environment and switch back the caller */
if (!sigsetjmp(self->env, 0)) {
- start_switch_fiber_asan(COROUTINE_YIELD, &fake_stack_save, leader.stack,
- leader.stack_size);
+ CoroutineUContext *leaderp = get_ptr_leader();
+
+ start_switch_fiber_asan(COROUTINE_YIELD, &fake_stack_save,
+ leaderp->stack, leaderp->stack_size);
start_switch_fiber_tsan(&fake_stack_save, self, true); /* true=caller */
siglongjmp(*(sigjmp_buf *)co->entry_arg, 1);
}
@@ -297,7 +301,7 @@ qemu_coroutine_switch(Coroutine *from_, Coroutine *to_,
int ret;
void *fake_stack_save = NULL;
- current = to_;
+ set_current(to_);
ret = sigsetjmp(from->env, 0);
if (ret == 0) {
@@ -315,18 +319,24 @@ qemu_coroutine_switch(Coroutine *from_, Coroutine *to_,
Coroutine *qemu_coroutine_self(void)
{
- if (!current) {
- current = &leader.base;
+ Coroutine *self = get_current();
+ CoroutineUContext *leaderp = get_ptr_leader();
+
+ if (!self) {
+ self = &leaderp->base;
+ set_current(self);
}
#ifdef CONFIG_TSAN
- if (!leader.tsan_co_fiber) {
- leader.tsan_co_fiber = __tsan_get_current_fiber();
+ if (!leaderp->tsan_co_fiber) {
+ leaderp->tsan_co_fiber = __tsan_get_current_fiber();
}
#endif
- return current;
+ return self;
}
bool qemu_in_coroutine(void)
{
- return current && current->caller;
+ Coroutine *self = get_current();
+
+ return self && self->caller;
}

124
coroutine-use-QEMU_DEFINE_STATIC_CO_TLS.patch Normal file
View File

@ -0,0 +1,124 @@
From: Stefan Hajnoczi <stefanha@redhat.com>
Date: Mon, 7 Mar 2022 15:38:52 +0000
Subject: coroutine: use QEMU_DEFINE_STATIC_CO_TLS()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: ac387a08a9c9f6b36757da912f0339c25f421f90
Thread-Local Storage variables cannot be used directly from coroutine
code because the compiler may optimize TLS variable accesses across
qemu_coroutine_yield() calls. When the coroutine is re-entered from
another thread the TLS variables from the old thread must no longer be
used.
Use QEMU_DEFINE_STATIC_CO_TLS() for the current and leader variables.
The alloc_pool QSLIST needs a typedef so the return value of
get_ptr_alloc_pool() can be stored in a local variable.
One example of why this code is necessary: a coroutine that yields
before calling qemu_coroutine_create() to create another coroutine is
affected by the TLS issue.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220307153853.602859-3-stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
[DF: see: https://bugzilla.redhat.com/show_bug.cgi?id=1952483]
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
util/qemu-coroutine.c | 41 ++++++++++++++++++++++++-----------------
1 file changed, 24 insertions(+), 17 deletions(-)
diff --git a/util/qemu-coroutine.c b/util/qemu-coroutine.c
index c03b2422ff686e7138bf7bf0abba..f3e8300c8d2ec64fc16fa562a518 100644
--- a/util/qemu-coroutine.c
+++ b/util/qemu-coroutine.c
@@ -18,6 +18,7 @@
#include "qemu/atomic.h"
#include "qemu/coroutine.h"
#include "qemu/coroutine_int.h"
+#include "qemu/coroutine-tls.h"
#include "block/aio.h"
/** Initial batch size is 64, and is increased on demand */
@@ -29,17 +30,20 @@ enum {
static QSLIST_HEAD(, Coroutine) release_pool = QSLIST_HEAD_INITIALIZER(pool);
static unsigned int pool_batch_size = POOL_INITIAL_BATCH_SIZE;
static unsigned int release_pool_size;
-static __thread QSLIST_HEAD(, Coroutine) alloc_pool = QSLIST_HEAD_INITIALIZER(pool);
-static __thread unsigned int alloc_pool_size;
-static __thread Notifier coroutine_pool_cleanup_notifier;
+
+typedef QSLIST_HEAD(, Coroutine) CoroutineQSList;
+QEMU_DEFINE_STATIC_CO_TLS(CoroutineQSList, alloc_pool);
+QEMU_DEFINE_STATIC_CO_TLS(unsigned int, alloc_pool_size);
+QEMU_DEFINE_STATIC_CO_TLS(Notifier, coroutine_pool_cleanup_notifier);
static void coroutine_pool_cleanup(Notifier *n, void *value)
{
Coroutine *co;
Coroutine *tmp;
+ CoroutineQSList *alloc_pool = get_ptr_alloc_pool();
- QSLIST_FOREACH_SAFE(co, &alloc_pool, pool_next, tmp) {
- QSLIST_REMOVE_HEAD(&alloc_pool, pool_next);
+ QSLIST_FOREACH_SAFE(co, alloc_pool, pool_next, tmp) {
+ QSLIST_REMOVE_HEAD(alloc_pool, pool_next);
qemu_coroutine_delete(co);
}
}
@@ -49,27 +53,30 @@ Coroutine *qemu_coroutine_create(CoroutineEntry *entry, void *opaque)
Coroutine *co = NULL;
if (CONFIG_COROUTINE_POOL) {
- co = QSLIST_FIRST(&alloc_pool);
+ CoroutineQSList *alloc_pool = get_ptr_alloc_pool();
+
+ co = QSLIST_FIRST(alloc_pool);
if (!co) {
if (release_pool_size > qatomic_read(&pool_batch_size)) {
/* Slow path; a good place to register the destructor, too. */
- if (!coroutine_pool_cleanup_notifier.notify) {
- coroutine_pool_cleanup_notifier.notify = coroutine_pool_cleanup;
- qemu_thread_atexit_add(&coroutine_pool_cleanup_notifier);
+ Notifier *notifier = get_ptr_coroutine_pool_cleanup_notifier();
+ if (!notifier->notify) {
+ notifier->notify = coroutine_pool_cleanup;
+ qemu_thread_atexit_add(notifier);
}
/* This is not exact; there could be a little skew between
* release_pool_size and the actual size of release_pool. But
* it is just a heuristic, it does not need to be perfect.
*/
- alloc_pool_size = qatomic_xchg(&release_pool_size, 0);
- QSLIST_MOVE_ATOMIC(&alloc_pool, &release_pool);
- co = QSLIST_FIRST(&alloc_pool);
+ set_alloc_pool_size(qatomic_xchg(&release_pool_size, 0));
+ QSLIST_MOVE_ATOMIC(alloc_pool, &release_pool);
+ co = QSLIST_FIRST(alloc_pool);
}
}
if (co) {
- QSLIST_REMOVE_HEAD(&alloc_pool, pool_next);
- alloc_pool_size--;
+ QSLIST_REMOVE_HEAD(alloc_pool, pool_next);
+ set_alloc_pool_size(get_alloc_pool_size() - 1);
}
}
@@ -93,9 +100,9 @@ static void coroutine_delete(Coroutine *co)
qatomic_inc(&release_pool_size);
return;
}
- if (alloc_pool_size < qatomic_read(&pool_batch_size)) {
- QSLIST_INSERT_HEAD(&alloc_pool, co, pool_next);
- alloc_pool_size++;
+ if (get_alloc_pool_size() < qatomic_read(&pool_batch_size)) {
+ QSLIST_INSERT_HEAD(get_ptr_alloc_pool(), co, pool_next);
+ set_alloc_pool_size(get_alloc_pool_size() + 1);
return;
}
}

89
coroutine-win32-use-QEMU_DEFINE_STATIC_C.patch Normal file
View File

@ -0,0 +1,89 @@
From: Stefan Hajnoczi <stefanha@redhat.com>
Date: Mon, 7 Mar 2022 15:38:53 +0000
Subject: coroutine-win32: use QEMU_DEFINE_STATIC_CO_TLS()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: c1fe694357a328c807ae3cc6961c19e923448fcc
Thread-Local Storage variables cannot be used directly from coroutine
code because the compiler may optimize TLS variable accesses across
qemu_coroutine_yield() calls. When the coroutine is re-entered from
another thread the TLS variables from the old thread must no longer be
used.
Use QEMU_DEFINE_STATIC_CO_TLS() for the current and leader variables.
I think coroutine-win32.c could get away with __thread because the
variables are only used in situations where either the stale value is
correct (current) or outside coroutine context (loading leader when
current is NULL). Due to the difficulty of being sure that this is
really safe in all scenarios it seems worth converting it anyway.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220307153853.602859-4-stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
[DF: see: https://bugzilla.redhat.com/show_bug.cgi?id=1952483]
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
util/coroutine-win32.c | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/util/coroutine-win32.c b/util/coroutine-win32.c
index de6bd4fd3e4e9747b08348e2a5f7..c02a62c896908162160440cbc8ba 100644
--- a/util/coroutine-win32.c
+++ b/util/coroutine-win32.c
@@ -25,6 +25,7 @@
#include "qemu/osdep.h"
#include "qemu-common.h"
#include "qemu/coroutine_int.h"
+#include "qemu/coroutine-tls.h"
typedef struct
{
@@ -34,8 +35,8 @@ typedef struct
CoroutineAction action;
} CoroutineWin32;
-static __thread CoroutineWin32 leader;
-static __thread Coroutine *current;
+QEMU_DEFINE_STATIC_CO_TLS(CoroutineWin32, leader);
+QEMU_DEFINE_STATIC_CO_TLS(Coroutine *, current);
/* This function is marked noinline to prevent GCC from inlining it
* into coroutine_trampoline(). If we allow it to do that then it
@@ -52,7 +53,7 @@ qemu_coroutine_switch(Coroutine *from_, Coroutine *to_,
CoroutineWin32 *from = DO_UPCAST(CoroutineWin32, base, from_);
CoroutineWin32 *to = DO_UPCAST(CoroutineWin32, base, to_);
- current = to_;
+ set_current(to_);
to->action = action;
SwitchToFiber(to->fiber);
@@ -89,14 +90,21 @@ void qemu_coroutine_delete(Coroutine *co_)
Coroutine *qemu_coroutine_self(void)
{
+ Coroutine *current = get_current();
+
if (!current) {
- current = &leader.base;
- leader.fiber = ConvertThreadToFiber(NULL);
+ CoroutineWin32 *leader = get_ptr_leader();
+
+ current = &leader->base;
+ set_current(current);
+ leader->fiber = ConvertThreadToFiber(NULL);
}
return current;
}
bool qemu_in_coroutine(void)
{
+ Coroutine *current = get_current();
+
return current && current->caller;
}

37
display-qxl-render-fix-race-condition-in.patch
View File

@ -1,37 +0,0 @@
From: Mauro Matteo Cascella <mcascell@redhat.com>
Date: Thu, 7 Apr 2022 10:11:06 +0200
Subject: display/qxl-render: fix race condition in qxl_cursor (CVE-2021-4207)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 9569f5cb5b4bffa9d3ebc8ba7da1e03830a9a895
References: bsc#1198037, CVE-2021-4207
Avoid fetching 'width' and 'height' a second time to prevent possible
race condition. Refer to security advisory
https://starlabs.sg/advisories/22-4207/ for more information.
Fixes: CVE-2021-4207
Signed-off-by: Mauro Matteo Cascella <mcascell@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20220407081106.343235-1-mcascell@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
hw/display/qxl-render.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hw/display/qxl-render.c b/hw/display/qxl-render.c
index d28849b121763600d21962321380..237ed293baaea76e9602e50a97ee 100644
--- a/hw/display/qxl-render.c
+++ b/hw/display/qxl-render.c
@@ -266,7 +266,7 @@ static QEMUCursor *qxl_cursor(PCIQXLDevice *qxl, QXLCursor *cursor,
}
break;
case SPICE_CURSOR_TYPE_ALPHA:
- size = sizeof(uint32_t) * cursor->header.width * cursor->header.height;
+ size = sizeof(uint32_t) * c->width * c->height;
qxl_unpack_chunks(c->data, size, qxl, &cursor->chunk, group_id);
if (qxl->debug > 2) {
cursor_print_ascii_art(c, "qxl/alpha");

67
doc-Add-the-SGX-numa-description.patch
View File

@ -1,67 +0,0 @@
From: Yang Zhong <yang.zhong@intel.com>
Date: Mon, 1 Nov 2021 12:20:08 -0400
Subject: doc: Add the SGX numa description
Git-commit: d1889b36098c79e2e6ac90faf3d0dc5ec0057677
References: bsc#1197807
Add the SGX numa reference command and how to check if
SGX numa is support or not with multiple EPC sections.
Signed-off-by: Yang Zhong <yang.zhong@intel.com>
Message-Id: <20211101162009.62161-5-yang.zhong@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
docs/system/i386/sgx.rst | 31 +++++++++++++++++++++++++++----
1 file changed, 27 insertions(+), 4 deletions(-)
diff --git a/docs/system/i386/sgx.rst b/docs/system/i386/sgx.rst
index f8fade5ac2d44b7378a87e8da14d..0f0a73f7587c44e69e028f061fed 100644
--- a/docs/system/i386/sgx.rst
+++ b/docs/system/i386/sgx.rst
@@ -141,8 +141,7 @@ To launch a SGX guest:
|qemu_system_x86| \\
-cpu host,+sgx-provisionkey \\
-object memory-backend-epc,id=mem1,size=64M,prealloc=on \\
- -object memory-backend-epc,id=mem2,size=28M \\
- -M sgx-epc.0.memdev=mem1,sgx-epc.1.memdev=mem2
+ -M sgx-epc.0.memdev=mem1,sgx-epc.0.node=0
Utilizing SGX in the guest requires a kernel/OS with SGX support.
The support can be determined in guest by::
@@ -152,8 +151,32 @@ The support can be determined in guest by::
and SGX epc info by::
$ dmesg | grep sgx
- [ 1.242142] sgx: EPC section 0x180000000-0x181bfffff
- [ 1.242319] sgx: EPC section 0x181c00000-0x1837fffff
+ [ 0.182807] sgx: EPC section 0x140000000-0x143ffffff
+ [ 0.183695] sgx: [Firmware Bug]: Unable to map EPC section to online node. Fallback to the NUMA node 0.
+
+To launch a SGX numa guest:
+
+.. parsed-literal::
+
+ |qemu_system_x86| \\
+ -cpu host,+sgx-provisionkey \\
+ -object memory-backend-ram,size=2G,host-nodes=0,policy=bind,id=node0 \\
+ -object memory-backend-epc,id=mem0,size=64M,prealloc=on,host-nodes=0,policy=bind \\
+ -numa node,nodeid=0,cpus=0-1,memdev=node0 \\
+ -object memory-backend-ram,size=2G,host-nodes=1,policy=bind,id=node1 \\
+ -object memory-backend-epc,id=mem1,size=28M,prealloc=on,host-nodes=1,policy=bind \\
+ -numa node,nodeid=1,cpus=2-3,memdev=node1 \\
+ -M sgx-epc.0.memdev=mem0,sgx-epc.0.node=0,sgx-epc.1.memdev=mem1,sgx-epc.1.node=1
+
+and SGX epc numa info by::
+
+ $ dmesg | grep sgx
+ [ 0.369937] sgx: EPC section 0x180000000-0x183ffffff
+ [ 0.370259] sgx: EPC section 0x184000000-0x185bfffff
+
+ $ dmesg | grep SRAT
+ [ 0.009981] ACPI: SRAT: Node 0 PXM 0 [mem 0x180000000-0x183ffffff]
+ [ 0.009982] ACPI: SRAT: Node 1 PXM 1 [mem 0x184000000-0x185bfffff]
References
----------

2
doc-add-our-support-doc-to-the-main-proj.patch
View File

@ -2,6 +2,8 @@ From: Bruce Rogers <brogers@suse.com>
Date: Fri, 29 Jan 2021 20:06:16 -0700
Subject: doc: add our support doc to the main project doc for x86 and s390x
Git-commit: 0000000000000000000000000000000000000000
Include-If: %if %{legacy_qemu_kvm}
Signed-off-by: Bruce Rogers <brogers@suse.com>

44
hostmem-default-the-amount-of-prealloc-t.patch Normal file
View File

@ -0,0 +1,44 @@
From: Jaroslav Jindrak <dzejrou@gmail.com>
Date: Tue, 17 May 2022 14:38:58 +0200
Subject: hostmem: default the amount of prealloc-threads to smp-cpus
Git-commit: f8d426a6852c560fdd8648ae961c8189909a4b82
References: bsc#1197084
Prior to the introduction of the prealloc-threads property, the amount
of threads used to preallocate memory was derived from the value of
smp-cpus passed to qemu, the amount of physical cpus of the host
and a hardcoded maximum value. When the prealloc-threads property
was introduced, it included a default of 1 in backends/hostmem.c and
a default of smp-cpus using the sugar API for the property itself. The
latter default is not used when the property is not specified on qemu's
command line, so guests that were not adjusted for this change suddenly
started to use the default of 1 thread to preallocate memory, which
resulted in observable slowdowns in guest boots for guests with large
memory (e.g. when using libvirt <8.2.0 or managing guests manually).
This commit restores the original behavior for these cases while not
impacting guests started with the prealloc-threads property in any way.
Fixes: 220c1fd864e9d ("hostmem: introduce "prealloc-threads" property")
Signed-off-by: Jaroslav Jindrak <dzejrou@gmail.com>
Message-Id: <20220517123858.7933-1-dzejrou@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
backends/hostmem.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/backends/hostmem.c b/backends/hostmem.c
index b2a5e905e866cd52f2f3e84f7d91..e4062a1c34920b5302fe57b8fdac 100644
--- a/backends/hostmem.c
+++ b/backends/hostmem.c
@@ -274,7 +274,7 @@ static void host_memory_backend_init(Object *obj)
backend->merge = machine_mem_merge(machine);
backend->dump = machine_dump_guest_core(machine);
backend->reserve = true;
- backend->prealloc_threads = 1;
+ backend->prealloc_threads = machine->smp.cpus;
}
static void host_memory_backend_post_init(Object *obj)

64
hw-i386-amd_iommu-Fix-maybe-uninitialize.patch
View File

@ -1,64 +0,0 @@
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Mon, 21 Mar 2022 15:33:46 +0100
Subject: hw/i386/amd_iommu: Fix maybe-uninitialized error with GCC 12
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 00000000000000000000000000000000000000000000
References: bsc#1197018
Be more explicit that the loop must roll at least once. Avoids the
following warning:
FAILED: libqemu-x86_64-softmmu.fa.p/hw_i386_amd_iommu.c.o
In function 'pte_get_page_mask',
inlined from 'amdvi_page_walk' at hw/i386/amd_iommu.c:945:25,
inlined from 'amdvi_do_translate' at hw/i386/amd_iommu.c:989:5,
inlined from 'amdvi_translate' at hw/i386/amd_iommu.c:1038:5:
hw/i386/amd_iommu.c:877:38: error: 'oldlevel' may be used uninitialized [-Werror=maybe-uninitialized]
877 | return ~((1UL << ((oldlevel * 9) + 3)) - 1);
| ~~~~~~~~~~~~~~~~^~~~
hw/i386/amd_iommu.c: In function 'amdvi_translate':
hw/i386/amd_iommu.c:906:41: note: 'oldlevel' was declared here
906 | unsigned level, present, pte_perms, oldlevel;
| ^~~~~~~~
cc1: all warnings being treated as errors
Having:
$ gcc --version
gcc (Debian 12-20220313-1) 12.0.1 20220314 (experimental)
Reported-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
hw/i386/amd_iommu.c | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/hw/i386/amd_iommu.c b/hw/i386/amd_iommu.c
index 91fe34ae5899339e4114a6e0cac6..f2960a0f381298c8c64009043ab2 100644
--- a/hw/i386/amd_iommu.c
+++ b/hw/i386/amd_iommu.c
@@ -911,7 +911,7 @@ static void amdvi_page_walk(AMDVIAddressSpace *as, uint64_t *dte,
}
/* we are at the leaf page table or page table encodes a huge page */
- while (level > 0) {
+ do {
pte_perms = amdvi_get_perms(pte);
present = pte & 1;
if (!present || perms != (perms & pte_perms)) {
@@ -930,10 +930,7 @@ static void amdvi_page_walk(AMDVIAddressSpace *as, uint64_t *dte,
}
oldlevel = level;
level = get_pte_translation_mode(pte);
- if (level == 0x7) {
- break;
- }
- }
+ } while (level > 0 && level < 7);
if (level == 0x7) {
page_mask = pte_override_page_mask(pte);

57
hw-intc-exynos4210_gic-provide-more-room.patch
View File

@ -1,57 +0,0 @@
From: Bruce Rogers <brogers@suse.com>
Date: Wed, 15 May 2019 13:32:01 -0600
Subject: hw/intc/exynos4210_gic: provide more room when formatting alias names
sprintf related parameter validation complains about the size of the
buffer being written to in exynos4210_gic_realize(). Provide a bit more
space to avoid the following warning:
/home/abuild/rpmbuild/BUILD/qemu-4.0.0/hw/intc/exynos4210_gic.c: In function 'exynos4210_gic_realize':
/home/abuild/rpmbuild/BUILD/qemu-4.0.0/hw/intc/exynos4210_gic.c:316:36: error: '%x' directive writing between 1 and 7 bytes into a region of size between 4 and 28 [-Werror=format-overflow=]
316 | sprintf(cpu_alias_name, "%s%x", cpu_prefix, i);
| ^~
/home/abuild/rpmbuild/BUILD/qemu-4.0.0/hw/intc/exynos4210_gic.c:316:33: note: directive argument in the range [0, 29020050]
316 | sprintf(cpu_alias_name, "%s%x", cpu_prefix, i);
| ^~~~~~
In file included from /usr/include/stdio.h:867,
from /home/abuild/rpmbuild/BUILD/qemu-4.0.0/include/qemu/osdep.h:99,
from /home/abuild/rpmbuild/BUILD/qemu-4.0.0/hw/intc/exynos4210_gic.c:23:
/usr/include/bits/stdio2.h:36:10: note: '__builtin___sprintf_chk' output between 2 and 32 bytes into a destination of size 28
36 | return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
37 | __bos (__s), __fmt, __va_arg_pack ());
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/abuild/rpmbuild/BUILD/qemu-4.0.0/hw/intc/exynos4210_gic.c:326:37: error: '%x' directive writing between 1 and 7 bytes into a region of size between 3 and 28 [-Werror=format-overflow=]
326 | sprintf(dist_alias_name, "%s%x", dist_prefix, i);
| ^~
/home/abuild/rpmbuild/BUILD/qemu-4.0.0/hw/intc/exynos4210_gic.c:326:34: note: directive argument in the range [0, 29020050]
326 | sprintf(dist_alias_name, "%s%x", dist_prefix, i);
| ^~~~~~
In file included from /usr/include/stdio.h:867,
from /home/abuild/rpmbuild/BUILD/qemu-4.0.0/include/qemu/osdep.h:99,
from /home/abuild/rpmbuild/BUILD/qemu-4.0.0/hw/intc/exynos4210_gic.c:23:
/usr/include/bits/stdio2.h:36:10: note: '__builtin___sprintf_chk' output between 2 and 33 bytes into a destination of size 28
36 | return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
37 | __bos (__s), __fmt, __va_arg_pack ());
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Signed-off-by: Bruce Rogers <brogers@suse.com>
---
hw/intc/exynos4210_gic.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/hw/intc/exynos4210_gic.c b/hw/intc/exynos4210_gic.c
index bc73d1f1152438de04c252a2ec84..db58021ef62cdf8554d7e00e0761 100644
--- a/hw/intc/exynos4210_gic.c
+++ b/hw/intc/exynos4210_gic.c
@@ -291,8 +291,8 @@ static void exynos4210_gic_realize(DeviceState *dev, Error **errp)
SysBusDevice *sbd = SYS_BUS_DEVICE(obj);
const char cpu_prefix[] = "exynos4210-gic-alias_cpu";
const char dist_prefix[] = "exynos4210-gic-alias_dist";
- char cpu_alias_name[sizeof(cpu_prefix) + 3];
- char dist_alias_name[sizeof(cpu_prefix) + 3];
+ char cpu_alias_name[sizeof(cpu_prefix) + 7];
+ char dist_alias_name[sizeof(cpu_prefix) + 8];
SysBusDevice *gicbusdev;
uint32_t n = s->num_cpu;
uint32_t i;

65
hw-nvme-fix-CVE-2021-3929.patch
View File

@ -1,65 +0,0 @@
From: Klaus Jensen <k.jensen@samsung.com>
Date: Fri, 17 Dec 2021 10:44:01 +0100
Subject: hw/nvme: fix CVE-2021-3929
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 736b01642d85be832385063f278fe7cd4ffb5221
Refrences: bsc#1193880 CVE-2021-3929
This fixes CVE-2021-3929 "locally" by denying DMA to the iomem of the
device itself. This still allows DMA to MMIO regions of other devices
(e.g. doing P2P DMA to the controller memory buffer of another NVMe
device).
Fixes: CVE-2021-3929
Reported-by: Qiuhao Li <Qiuhao.Li@outlook.com>
Reviewed-by: Keith Busch <kbusch@kernel.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Klaus Jensen <k.jensen@samsung.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
hw/nvme/ctrl.c | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c
index 5f573c417b3d66c30814a74b192a..eda52c6ac74b3419ca4b656d0ee0 100644
--- a/hw/nvme/ctrl.c
+++ b/hw/nvme/ctrl.c
@@ -357,6 +357,24 @@ static inline void *nvme_addr_to_pmr(NvmeCtrl *n, hwaddr addr)
return memory_region_get_ram_ptr(&n->pmr.dev->mr) + (addr - n->pmr.cba);
}
+static inline bool nvme_addr_is_iomem(NvmeCtrl *n, hwaddr addr)
+{
+ hwaddr hi, lo;
+
+ /*
+ * The purpose of this check is to guard against invalid "local" access to
+ * the iomem (i.e. controller registers). Thus, we check against the range
+ * covered by the 'bar0' MemoryRegion since that is currently composed of
+ * two subregions (the NVMe "MBAR" and the MSI-X table/pba). Note, however,
+ * that if the device model is ever changed to allow the CMB to be located
+ * in BAR0 as well, then this must be changed.
+ */
+ lo = n->bar0.addr;
+ hi = lo + int128_get64(n->bar0.size);
+
+ return addr >= lo && addr < hi;
+}
+
static int nvme_addr_read(NvmeCtrl *n, hwaddr addr, void *buf, int size)
{
hwaddr hi = addr + size - 1;
@@ -614,6 +632,10 @@ static uint16_t nvme_map_addr(NvmeCtrl *n, NvmeSg *sg, hwaddr addr, size_t len)
trace_pci_nvme_map_addr(addr, len);
+ if (nvme_addr_is_iomem(n, addr)) {
+ return NVME_DATA_TRAS_ERROR;
+ }
+
if (nvme_addr_is_cmb(n, addr)) {
cmb = true;
} else if (nvme_addr_is_pmr(n, addr)) {

52
hw-nvram-at24-return-0xff-if-1-byte-addr.patch
View File

@ -1,52 +0,0 @@
From: Patrick Venture <venture@google.com>
Date: Mon, 20 Dec 2021 13:21:37 -0800
Subject: hw/nvram: at24 return 0xff if 1 byte address
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 1cbab82e9d1bdb2c7b9ef46a396fdc03ea3fa04c
The at24 eeproms are 2 byte devices that return 0xff when they are read
from with a partial (1-byte) address written. This distinction was
found comparing model behavior to real hardware testing.
Tested: `i2ctransfer -f -y 45 w1@85 0 r1` returns 0xff instead of next
byte
Signed-off-by: Patrick Venture <venture@google.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20211220212137.1244511-1-venture@google.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
hw/nvram/eeprom_at24c.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/hw/nvram/eeprom_at24c.c b/hw/nvram/eeprom_at24c.c
index af6f5dbb998475871b4defb59c47..b956b8e2b2d9d74a1bc61bc16eb9 100644
--- a/hw/nvram/eeprom_at24c.c
+++ b/hw/nvram/eeprom_at24c.c
@@ -58,9 +58,10 @@ int at24c_eeprom_event(I2CSlave *s, enum i2c_event event)
switch (event) {
case I2C_START_SEND:
- case I2C_START_RECV:
case I2C_FINISH:
ee->haveaddr = 0;
+ /* fallthrough */
+ case I2C_START_RECV:
DPRINTK("clear\n");
if (ee->blk && ee->changed) {
int len = blk_pwrite(ee->blk, 0, ee->mem, ee->rsize, 0);
@@ -84,6 +85,10 @@ uint8_t at24c_eeprom_recv(I2CSlave *s)
EEPROMState *ee = AT24C_EE(s);
uint8_t ret;
+ if (ee->haveaddr == 1) {
+ return 0xff;
+ }
+
ret = ee->mem[ee->cur];
ee->cur = (ee->cur + 1u) % ee->rsize;

6
hw-scsi-megasas-check-for-NULL-frame-in-.patch
View File

@ -2,7 +2,7 @@ From: Mauro Matteo Cascella <mcascell@redhat.com>
Date: Mon, 7 Mar 2022 16:22:01 +0100
Subject: hw/scsi/megasas: check for NULL frame in megasas_command_cancelled()
Git-commit: 00000000000000000000000000000000000000000000
Git-commit: 0000000000000000000000000000000000000000
References: bsc#1180432, CVE-2020-35503
Ensure that 'cmd->frame' is not NULL before accessing the 'header' field.
@ -17,10 +17,10 @@ Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hw/scsi/megasas.c b/hw/scsi/megasas.c
index 4ff51221d4cd0952d9394b8f66b1..07b2b08bf2866907acf0a2b4450c 100644
index d5dfb412bac4251c9fa06eb416b4..cf97b4d2db5bf91a0e18908e900d 100644
--- a/hw/scsi/megasas.c
+++ b/hw/scsi/megasas.c
@@ -1891,7 +1891,7 @@ static void megasas_command_cancelled(SCSIRequest *req)
@@ -1927,7 +1927,7 @@ static void megasas_command_cancelled(SCSIRequest *req)
{
MegasasCmd *cmd = req->hba_private;

10
hw-smbios-handle-both-file-formats-regar.patch
View File

@ -16,10 +16,10 @@ Signed-off-by: Bruce Rogers <brogers@suse.com>
1 file changed, 39 insertions(+), 4 deletions(-)
diff --git a/hw/smbios/smbios.c b/hw/smbios/smbios.c
index 7397e567373b2dbfabae8fe10a5f..61b7546a6dd7010bdc248f969100 100644
index 60349ee402719d55896658bf203c..59091c8454bf6d8d05c4c8313753 100644
--- a/hw/smbios/smbios.c
+++ b/hw/smbios/smbios.c
@@ -1138,6 +1138,7 @@ void smbios_entry_add(QemuOpts *opts, Error **errp)
@@ -1180,6 +1180,7 @@ void smbios_entry_add(QemuOpts *opts, Error **errp)
struct smbios_structure_header *header;
int size;
struct smbios_table *table; /* legacy mode only */
@ -27,7 +27,7 @@ index 7397e567373b2dbfabae8fe10a5f..61b7546a6dd7010bdc248f969100 100644
if (!qemu_opts_validate(opts, qemu_smbios_file_opts, errp)) {
return;
@@ -1150,11 +1151,21 @@ void smbios_entry_add(QemuOpts *opts, Error **errp)
@@ -1192,11 +1193,21 @@ void smbios_entry_add(QemuOpts *opts, Error **errp)
}
/*
@ -53,7 +53,7 @@ index 7397e567373b2dbfabae8fe10a5f..61b7546a6dd7010bdc248f969100 100644
header = (struct smbios_structure_header *)(smbios_tables +
smbios_tables_len);
@@ -1169,6 +1180,19 @@ void smbios_entry_add(QemuOpts *opts, Error **errp)
@@ -1211,6 +1222,19 @@ void smbios_entry_add(QemuOpts *opts, Error **errp)
header->type);
return;
}
@ -73,7 +73,7 @@ index 7397e567373b2dbfabae8fe10a5f..61b7546a6dd7010bdc248f969100 100644
set_bit(header->type, have_binfile_bitmap);
if (header->type == 4) {
@@ -1189,6 +1213,17 @@ void smbios_entry_add(QemuOpts *opts, Error **errp)
@@ -1231,6 +1255,17 @@ void smbios_entry_add(QemuOpts *opts, Error **errp)
* delete the one we don't need from smbios_set_defaults(),
* once we know which machine version has been requested.
*/

4
hw-usb-hcd-ehci-fix-writeback-order.patch
View File

@ -43,10 +43,10 @@ Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/hw/usb/hcd-ehci.c b/hw/usb/hcd-ehci.c
index 6caa7ac6c28f56416f652b665787..3464b2406e94dcc7272116c7249d 100644
index 33a8a377bd9503cf0ac1e6bd97e6..d4da8dcb8d15e3f1dcb8b6cfddf7 100644
--- a/hw/usb/hcd-ehci.c
+++ b/hw/usb/hcd-ehci.c
@@ -2009,7 +2009,10 @@ static int ehci_state_writeback(EHCIQueue *q)
@@ -2011,7 +2011,10 @@ static int ehci_state_writeback(EHCIQueue *q)
ehci_trace_qtd(q, NLPTR_GET(p->qtdaddr), (EHCIqtd *) &q->qh.next_qtd);
qtd = (uint32_t *) &q->qh.next_qtd;
addr = NLPTR_GET(p->qtdaddr);

41
increase-x86_64-physical-bits-to-42.patch
View File

@ -13,6 +13,47 @@ memory hole.
Signed-off-by: Bruce Rogers <brogers@suse.com>
Signed-off-by: Andreas Färber <afaerber@suse.de>
The DSDT needs to be updated as well, or tests will fail.
The necessary changes are the following ones:
@@ -5,13 +5,13 @@
*
* Disassembling to symbolic ASL+ operators
*
- * Disassembly of /tmp/aml-ML87L1, Thu May 19 16:20:30 2022
+ * Disassembly of tests/data/acpi/microvm/DSDT.pcie, Thu May 19 16:20:30 2022
*
* Original Table Header:
* Signature "DSDT"
* Length 0x00000BCF (3023)
* Revision 0x02
- * Checksum 0xD1
+ * Checksum 0xD8
* OEM ID "BOCHS "
* OEM Table ID "BXPC "
* OEM Revision 0x00000001 (1)
@@ -1256,10 +1256,10 @@
,, , AddressRangeMemory, TypeStatic)
QWordMemory (ResourceProducer, PosDecode, MinFixed, MaxFixed, NonCacheable, ReadWrite,
0x0000000000000000, // Granularity
- 0x0000030000000000, // Range Minimum
- 0x000003FFFFFFFFFF, // Range Maximum
+ 0x000000C000000000, // Range Minimum
+ 0x000000FFFFFFFFFF, // Range Maximum
0x0000000000000000, // Translation Offset
- 0x0000010000000000, // Length
+ 0x0000004000000000, // Length
,, , AddressRangeMemory, TypeStatic)
})
Name (SUPP, Zero)
We cannot, however, commit the modified binary as part of this patch,
will not apply it. The binary is instead put in the package, and copied
in place in the spec file, just before 'make check'.
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
target/i386/tcg/helper-tcg.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

58
intc-exynos4210_gic-replace-snprintf-wit.patch Normal file
View File

@ -0,0 +1,58 @@
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Wed, 20 Apr 2022 17:25:58 +0400
Subject: intc/exynos4210_gic: replace snprintf() with g_strdup_printf()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: a0e04317f3ef86d7b18ec6e6459d8aa66f787d12
While at it, replace '%x' with '%u' as suggested by Philippe Mathieu-Daudé.
Also fixes a GCC 12.0.1 -Wformat-overflow false-positive.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220420132624.2439741-16-marcandre.lureau@redhat.com>
[DF: Replaces downstream patch: hw-intc-exynos4210_gic-provide-more-room.patch]
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
hw/intc/exynos4210_gic.c | 9 +++------
1 file changed, 3 insertions(+), 6 deletions(-)
diff --git a/hw/intc/exynos4210_gic.c b/hw/intc/exynos4210_gic.c
index bc73d1f1152438de04c252a2ec84..f8dd719caa1e598a32ab6cc6ee7a 100644
--- a/hw/intc/exynos4210_gic.c
+++ b/hw/intc/exynos4210_gic.c
@@ -289,10 +289,6 @@ static void exynos4210_gic_realize(DeviceState *dev, Error **errp)
Object *obj = OBJECT(dev);
Exynos4210GicState *s = EXYNOS4210_GIC(obj);
SysBusDevice *sbd = SYS_BUS_DEVICE(obj);
- const char cpu_prefix[] = "exynos4210-gic-alias_cpu";
- const char dist_prefix[] = "exynos4210-gic-alias_dist";
- char cpu_alias_name[sizeof(cpu_prefix) + 3];
- char dist_alias_name[sizeof(cpu_prefix) + 3];
SysBusDevice *gicbusdev;
uint32_t n = s->num_cpu;
uint32_t i;
@@ -322,8 +318,10 @@ static void exynos4210_gic_realize(DeviceState *dev, Error **errp)
*/
assert(n <= EXYNOS4210_NCPUS);
for (i = 0; i < n; i++) {
+ g_autofree char *cpu_alias_name = g_strdup_printf("exynos4210-gic-alias_cpu%u", i);
+ g_autofree char *dist_alias_name = g_strdup_printf("exynos4210-gic-alias_dist%u", i);
+
/* Map CPU interface per SMP Core */
- sprintf(cpu_alias_name, "%s%x", cpu_prefix, i);
memory_region_init_alias(&s->cpu_alias[i], obj,
cpu_alias_name,
sysbus_mmio_get_region(gicbusdev, 1),
@@ -333,7 +331,6 @@ static void exynos4210_gic_realize(DeviceState *dev, Error **errp)
EXYNOS4210_EXT_GIC_CPU_GET_OFFSET(i), &s->cpu_alias[i]);
/* Map Distributor per SMP Core */
- sprintf(dist_alias_name, "%s%x", dist_prefix, i);
memory_region_init_alias(&s->dist_alias[i], obj,
dist_alias_name,
sysbus_mmio_get_region(gicbusdev, 0),

78
iotest-065-explicit-compression-type.patch
View File

@ -1,78 +0,0 @@
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Thu, 23 Dec 2021 17:01:33 +0100
Subject: iotest 065: explicit compression type
Git-commit: 12a936171d71f839dc907fff7887358a05ac20f8
The test checks different options. It of course fails if set
IMGOPTS='compression_type=zstd'. So, let's be explicit in what
compression type we want and independent of IMGOPTS. Test both existing
compression types.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223160144.1097696-9-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
tests/qemu-iotests/065 | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/tests/qemu-iotests/065 b/tests/qemu-iotests/065
index dc7716275f0d5e8b6aeb572244fb..f7c1b68dadbaf151781dfc031166 100755
--- a/tests/qemu-iotests/065
+++ b/tests/qemu-iotests/065
@@ -88,7 +88,7 @@ class TestQMP(TestImageInfoSpecific):
class TestQCow2(TestQemuImgInfo):
'''Testing a qcow2 version 2 image'''
- img_options = 'compat=0.10'
+ img_options = 'compat=0.10,compression_type=zlib'
json_compare = { 'compat': '0.10', 'refcount-bits': 16,
'compression-type': 'zlib' }
human_compare = [ 'compat: 0.10', 'compression type: zlib',
@@ -96,17 +96,17 @@ class TestQCow2(TestQemuImgInfo):
class TestQCow3NotLazy(TestQemuImgInfo):
'''Testing a qcow2 version 3 image with lazy refcounts disabled'''
- img_options = 'compat=1.1,lazy_refcounts=off'
+ img_options = 'compat=1.1,lazy_refcounts=off,compression_type=zstd'
json_compare = { 'compat': '1.1', 'lazy-refcounts': False,
'refcount-bits': 16, 'corrupt': False,
- 'compression-type': 'zlib', 'extended-l2': False }
- human_compare = [ 'compat: 1.1', 'compression type: zlib',
+ 'compression-type': 'zstd', 'extended-l2': False }
+ human_compare = [ 'compat: 1.1', 'compression type: zstd',
'lazy refcounts: false', 'refcount bits: 16',
'corrupt: false', 'extended l2: false' ]
class TestQCow3Lazy(TestQemuImgInfo):
'''Testing a qcow2 version 3 image with lazy refcounts enabled'''
- img_options = 'compat=1.1,lazy_refcounts=on'
+ img_options = 'compat=1.1,lazy_refcounts=on,compression_type=zlib'
json_compare = { 'compat': '1.1', 'lazy-refcounts': True,
'refcount-bits': 16, 'corrupt': False,
'compression-type': 'zlib', 'extended-l2': False }
@@ -117,7 +117,7 @@ class TestQCow3Lazy(TestQemuImgInfo):
class TestQCow3NotLazyQMP(TestQMP):
'''Testing a qcow2 version 3 image with lazy refcounts disabled, opening
with lazy refcounts enabled'''
- img_options = 'compat=1.1,lazy_refcounts=off'
+ img_options = 'compat=1.1,lazy_refcounts=off,compression_type=zlib'
qemu_options = 'lazy-refcounts=on'
compare = { 'compat': '1.1', 'lazy-refcounts': False,
'refcount-bits': 16, 'corrupt': False,
@@ -127,11 +127,11 @@ class TestQCow3NotLazyQMP(TestQMP):
class TestQCow3LazyQMP(TestQMP):
'''Testing a qcow2 version 3 image with lazy refcounts enabled, opening
with lazy refcounts disabled'''
- img_options = 'compat=1.1,lazy_refcounts=on'
+ img_options = 'compat=1.1,lazy_refcounts=on,compression_type=zstd'
qemu_options = 'lazy-refcounts=off'
compare = { 'compat': '1.1', 'lazy-refcounts': True,
'refcount-bits': 16, 'corrupt': False,
- 'compression-type': 'zlib', 'extended-l2': False }
+ 'compression-type': 'zstd', 'extended-l2': False }
TestImageInfoSpecific = None
TestQemuImgInfo = None

32
iotest-214-explicit-compression-type.patch
View File

@ -1,32 +0,0 @@
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Thu, 23 Dec 2021 17:01:43 +0100
Subject: iotest 214: explicit compression type
Git-commit: da87d5f83a23dd9b252fabc7787383ce6d2454a3
The test-case "Corrupted size field in compressed cluster descriptor"
heavily depends on zlib compression type. So, make it explicit. This
way test passes with IMGOPTS='compression_type=zstd'.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20211223160144.1097696-19-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
tests/qemu-iotests/214 | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/qemu-iotests/214 b/tests/qemu-iotests/214
index 0889089d81cfa9375b5f4d30bfd5..c66e246ba248e221e59743b247e3 100755
--- a/tests/qemu-iotests/214
+++ b/tests/qemu-iotests/214
@@ -51,7 +51,7 @@ echo
# The L2 entries of the two compressed clusters are located at
# 0x800000 and 0x800008, their original values are 0x4008000000a00000
# and 0x4008000000a00802 (5 sectors for compressed data each).
-_make_test_img 8M -o cluster_size=2M
+_make_test_img 8M -o cluster_size=2M,compression_type=zlib
$QEMU_IO -c "write -c -P 0x11 0 2M" -c "write -c -P 0x11 2M 2M" "$TEST_IMG" \
2>&1 | _filter_qemu_io | _filter_testdir

82
iotest-302-use-img_info_log-helper.patch
View File

@ -1,82 +0,0 @@
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Thu, 23 Dec 2021 17:01:36 +0100
Subject: iotest 302: use img_info_log() helper
Git-commit: c30175d6fbc73a86a8013da195471d1a3490178f
Instead of qemu_img_log("info", ..) use generic helper img_info_log().
img_info_log() has smarter logic. For example it use filter_img_info()
to filter output, which in turns filter a compression type. So it will
help us in future when we implement a possibility to use zstd
compression by default (with help of some runtime config file or maybe
build option). For now to test you should recompile qemu with a small
addition into block/qcow2.c before
"if (qcow2_opts->has_compression_type":
if (!qcow2_opts->has_compression_type && version >= 3) {
qcow2_opts->has_compression_type = true;
qcow2_opts->compression_type = QCOW2_COMPRESSION_TYPE_ZSTD;
}
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20211223160144.1097696-12-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
tests/qemu-iotests/302 | 4 +++-
tests/qemu-iotests/302.out | 7 +++----
2 files changed, 6 insertions(+), 5 deletions(-)
diff --git a/tests/qemu-iotests/302 b/tests/qemu-iotests/302
index 5695af4914654f2a434391b6d31c..a6d79e727b55dd2d0d0e6fb28650 100755
--- a/tests/qemu-iotests/302
+++ b/tests/qemu-iotests/302
@@ -34,6 +34,7 @@ from iotests import (
qemu_img_measure,
qemu_io,
qemu_nbd_popen,
+ img_info_log,
)
iotests.script_initialize(supported_fmts=["qcow2"])
@@ -88,6 +89,7 @@ with tarfile.open(tar_file, "w") as tar:
tar_file):
iotests.log("=== Target image info ===")
+ # Not img_info_log as it enforces imgfmt, but now we print info on raw
qemu_img_log("info", nbd_uri)
qemu_img(
@@ -99,7 +101,7 @@ with tarfile.open(tar_file, "w") as tar:
nbd_uri)
iotests.log("=== Converted image info ===")
- qemu_img_log("info", nbd_uri)
+ img_info_log(nbd_uri)
iotests.log("=== Converted image check ===")
qemu_img_log("check", nbd_uri)
diff --git a/tests/qemu-iotests/302.out b/tests/qemu-iotests/302.out
index e2f6077e8330854905882a9175a8..3e7c281b9116e4b54a4d6834d9ef 100644
--- a/tests/qemu-iotests/302.out
+++ b/tests/qemu-iotests/302.out
@@ -6,14 +6,13 @@ virtual size: 448 KiB (458752 bytes)
disk size: unavailable
=== Converted image info ===
-image: nbd+unix:///exp?socket=SOCK_DIR/PID-nbd-sock
-file format: qcow2
+image: TEST_IMG
+file format: IMGFMT
virtual size: 1 GiB (1073741824 bytes)
-disk size: unavailable
cluster_size: 65536
Format specific information:
compat: 1.1
- compression type: zlib
+ compression type: COMPRESSION_TYPE
lazy refcounts: false
refcount bits: 16
corrupt: false

101
iotest-303-explicit-compression-type.patch
View File

@ -1,101 +0,0 @@
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Thu, 23 Dec 2021 17:01:32 +0100
Subject: iotest 303: explicit compression type
Git-commit: 677e0bae686e7c670a71d1f6a491a7f06f77de73
The test prints qcow2 header fields which depends on chosen compression
type. So, let's be explicit in what compression type we want and
independent of IMGOPTS. Test both existing compression types.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20211223160144.1097696-8-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
tests/qemu-iotests/303 | 25 ++++++++++++++++---------
tests/qemu-iotests/303.out | 30 +++++++++++++++++++++++++++++-
2 files changed, 45 insertions(+), 10 deletions(-)
diff --git a/tests/qemu-iotests/303 b/tests/qemu-iotests/303
index 475cb5428db4f047b7db63776ca0..16c2e1082768f67c14c8665b2b2c 100755
--- a/tests/qemu-iotests/303
+++ b/tests/qemu-iotests/303
@@ -54,12 +54,19 @@ def add_bitmap(num, begin, end, disabled):
log('')
-qemu_img_create('-f', iotests.imgfmt, disk, '10M')
-
-add_bitmap(1, 0, 6, False)
-add_bitmap(2, 6, 8, True)
-dump = ['./qcow2.py', disk, 'dump-header']
-subprocess.run(dump)
-# Dump the metadata in JSON format
-dump.append('-j')
-subprocess.run(dump)
+def test(compression_type: str, json_output: bool) -> None:
+ qemu_img_create('-f', iotests.imgfmt,
+ '-o', f'compression_type={compression_type}',
+ disk, '10M')
+ add_bitmap(1, 0, 6, False)
+ add_bitmap(2, 6, 8, True)
+
+ cmd = ['./qcow2.py', disk, 'dump-header']
+ if json_output:
+ cmd.append('-j')
+
+ subprocess.run(cmd)
+
+
+test('zlib', False)
+test('zstd', True)
diff --git a/tests/qemu-iotests/303.out b/tests/qemu-iotests/303.out
index 7c16998587840607918fd777fb99..b3c70827b74a253209527106a4a9 100644
--- a/tests/qemu-iotests/303.out
+++ b/tests/qemu-iotests/303.out
@@ -80,6 +80,34 @@ extra_data_size 0
Bitmap table type size offset
0 all-zeroes 0 0
+Add bitmap 1
+wrote 1048576/1048576 bytes at offset 0
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+wrote 1048576/1048576 bytes at offset 1048576
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+wrote 1048576/1048576 bytes at offset 2097152
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+wrote 1048576/1048576 bytes at offset 3145728
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+wrote 1048576/1048576 bytes at offset 4194304
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+wrote 1048576/1048576 bytes at offset 5242880
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+
+Add bitmap 2
+wrote 1048576/1048576 bytes at offset 6291456
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+wrote 1048576/1048576 bytes at offset 7340032
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+
{
"magic": 1363560955,
"version": 3,
@@ -94,7 +122,7 @@ Bitmap table type size offset
"refcount_table_clusters": 1,
"nb_snapshots": 0,
"snapshot_offset": 0,
- "incompatible_features": 0,
+ "incompatible_features": 8,
"compatible_features": 0,
"autoclear_features": 1,
"refcount_order": 4,

31
iotest-39-use-_qcow2_dump_header.patch
View File

@ -1,31 +0,0 @@
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Thu, 23 Dec 2021 17:01:40 +0100
Subject: iotest 39: use _qcow2_dump_header
Git-commit: 72be51ddb3d10974da5128ad20ee5ca7a13ddd54
_qcow2_dump_header has filter for compression type, so this change
makes test pass with IMGOPTS='compression_type=zstd'.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20211223160144.1097696-16-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
tests/qemu-iotests/039 | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/qemu-iotests/039 b/tests/qemu-iotests/039
index 8e783a8380666f26d237c7233602..00d379cde259d64a46e2a49d9f20 100755
--- a/tests/qemu-iotests/039
+++ b/tests/qemu-iotests/039
@@ -142,7 +142,7 @@ $QEMU_IMG commit "$TEST_IMG"
# The dirty bit must not be set
_qcow2_dump_header | grep incompatible_features
-$PYTHON qcow2.py "$TEST_IMG".base dump-header | grep incompatible_features
+_qcow2_dump_header "$TEST_IMG".base | grep incompatible_features
_check_test_img
TEST_IMG="$TEST_IMG".base _check_test_img

31
iotests-60-more-accurate-set-dirty-bit-i.patch
View File

@ -1,31 +0,0 @@
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Thu, 23 Dec 2021 17:01:42 +0100
Subject: iotests 60: more accurate set dirty bit in qcow2 header
Git-commit: 3a0e60a065ee32027684314a0d886d43bc6ce34b
Don't touch other incompatible bits, like compression-type. This makes
the test pass with IMGOPTS='compression_type=zstd'.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20211223160144.1097696-18-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
tests/qemu-iotests/060 | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060
index d1e3204d4ea42002263486eebafd..df87d600f761d254cff3f371794e 100755
--- a/tests/qemu-iotests/060
+++ b/tests/qemu-iotests/060
@@ -326,7 +326,7 @@ _make_test_img 64M
# Let the refblock appear unaligned
poke_file "$TEST_IMG" "$rt_offset" "\x00\x00\x00\x00\xff\xff\x2a\x00"
# Mark the image dirty, thus forcing an automatic check when opening it
-poke_file "$TEST_IMG" 72 "\x00\x00\x00\x00\x00\x00\x00\x01"
+$PYTHON qcow2.py "$TEST_IMG" set-feature-bit incompatible 0
# Open the image (qemu should refuse to do so)
$QEMU_IO -c close "$TEST_IMG" 2>&1 | _filter_testdir | _filter_imgfmt

61
iotests-MRCE-Write-data-to-source.patch
View File

@ -1,61 +0,0 @@
From: Hanna Reitz <hreitz@redhat.com>
Date: Thu, 23 Dec 2021 17:53:08 +0100
Subject: iotests/MRCE: Write data to source
Git-commit: fc2c3996a59683685a663deb3af12183ad24e4a7
This test assumes that mirror flushes the source when entering the READY
state, and that the format level will pass that flush on to the protocol
level (where we intercept it with blkdebug).
However, apparently that does not happen when using a VMDK image with
zeroed_grain=on, which actually is the default set by testenv.py. Right
now, Python tests ignore IMGOPTS, though, so this has no effect; but
Vladimir has a series that will change this, so we need to fix this test
before that series lands.
We can fix it by writing data to the source before we start the mirror
job; apparently that makes the (VMDK) format layer change its mind and
pass on the pre-READY flush to the protocol level, so the test passes
again. (I presume, without any data written, mirror just does a 64M
zero write on the target, which VMDK with zeroed_grain=on basically just
ignores.)
Without this, we do not get a flush, and so blkdebug only sees a single
flush at the end of the job instead of two, and therefore does not
inject an error, which makes the block job complete instead of raising
an error.
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223165308.103793-1-hreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
tests/qemu-iotests/tests/mirror-ready-cancel-error | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/tests/qemu-iotests/tests/mirror-ready-cancel-error b/tests/qemu-iotests/tests/mirror-ready-cancel-error
index f2dc88881f9e530db250d78e6f41..770ffca37930faf779b67166a806 100755
--- a/tests/qemu-iotests/tests/mirror-ready-cancel-error
+++ b/tests/qemu-iotests/tests/mirror-ready-cancel-error
@@ -36,6 +36,11 @@ class TestMirrorReadyCancelError(iotests.QMPTestCase):
assert iotests.qemu_img_create('-f', iotests.imgfmt, target,
str(image_size)) == 0
+ # Ensure that mirror will copy something before READY so the
+ # target format layer will forward the pre-READY flush to its
+ # file child
+ assert iotests.qemu_io_silent('-c', 'write -P 1 0 64k', source) == 0
+
self.vm = iotests.VM()
self.vm.launch()
@@ -97,7 +102,7 @@ class TestMirrorReadyCancelError(iotests.QMPTestCase):
# Write something so will not leave the job immediately, but
# flush first (which will fail, thanks to blkdebug)
res = self.vm.qmp('human-monitor-command',
- command_line='qemu-io mirror-top "write 0 64k"')
+ command_line='qemu-io mirror-top "write -P 2 0 64k"')
self.assert_qmp(res, 'return', '')
# Drain status change events

288
iotests-bash-tests-filter-compression-ty.patch
View File

@ -1,288 +0,0 @@
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Thu, 23 Dec 2021 17:01:41 +0100
Subject: iotests: bash tests: filter compression type
Git-commit: dba5aee4da9ce0a81e5a870c22a19926212dc87e
We want iotests pass with both the default zlib compression and with
IMGOPTS='compression_type=zstd'.
Actually the only test that is interested in real compression type in
test output is 287 (test for qcow2 compression type), so implement
specific option for it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223160144.1097696-17-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
tests/qemu-iotests/060.out | 2 +-
tests/qemu-iotests/061.out | 12 ++++++------
tests/qemu-iotests/082.out | 14 +++++++-------
tests/qemu-iotests/198.out | 4 ++--
tests/qemu-iotests/287 | 8 ++++----
tests/qemu-iotests/common.filter | 8 ++++++++
tests/qemu-iotests/common.rc | 14 +++++++++++++-
7 files changed, 41 insertions(+), 21 deletions(-)
diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out
index b74540bafbe84ff721fc0197d21c..329977d9b9b9fb4ec94615fff0a8 100644
--- a/tests/qemu-iotests/060.out
+++ b/tests/qemu-iotests/060.out
@@ -17,7 +17,7 @@ virtual size: 64 MiB (67108864 bytes)
cluster_size: 65536
Format specific information:
compat: 1.1
- compression type: zlib
+ compression type: COMPRESSION_TYPE
lazy refcounts: false
refcount bits: 16
corrupt: true
diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out
index 7ecbd4dea875ea26e0b131ec2922..139fc68177de57e33567094b84d6 100644
--- a/tests/qemu-iotests/061.out
+++ b/tests/qemu-iotests/061.out
@@ -525,7 +525,7 @@ virtual size: 64 MiB (67108864 bytes)
cluster_size: 65536
Format specific information:
compat: 1.1
- compression type: zlib
+ compression type: COMPRESSION_TYPE
lazy refcounts: false
refcount bits: 16
data file: TEST_DIR/t.IMGFMT.data
@@ -552,7 +552,7 @@ virtual size: 64 MiB (67108864 bytes)
cluster_size: 65536
Format specific information:
compat: 1.1
- compression type: zlib
+ compression type: COMPRESSION_TYPE
lazy refcounts: false
refcount bits: 16
data file: foo
@@ -567,7 +567,7 @@ virtual size: 64 MiB (67108864 bytes)
cluster_size: 65536
Format specific information:
compat: 1.1
- compression type: zlib
+ compression type: COMPRESSION_TYPE
lazy refcounts: false
refcount bits: 16
data file raw: false
@@ -583,7 +583,7 @@ virtual size: 64 MiB (67108864 bytes)
cluster_size: 65536
Format specific information:
compat: 1.1
- compression type: zlib
+ compression type: COMPRESSION_TYPE
lazy refcounts: false
refcount bits: 16
data file: TEST_DIR/t.IMGFMT.data
@@ -597,7 +597,7 @@ virtual size: 64 MiB (67108864 bytes)
cluster_size: 65536
Format specific information:
compat: 1.1
- compression type: zlib
+ compression type: COMPRESSION_TYPE
lazy refcounts: false
refcount bits: 16
data file: TEST_DIR/t.IMGFMT.data
@@ -612,7 +612,7 @@ virtual size: 64 MiB (67108864 bytes)
cluster_size: 65536
Format specific information:
compat: 1.1
- compression type: zlib
+ compression type: COMPRESSION_TYPE
lazy refcounts: false
refcount bits: 16
data file: TEST_DIR/t.IMGFMT.data
diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out
index 077ed0f2c7d933034a50c50c5de6..d0dd333117a571063a035d7d5c31 100644
--- a/tests/qemu-iotests/082.out
+++ b/tests/qemu-iotests/082.out
@@ -17,7 +17,7 @@ virtual size: 128 MiB (134217728 bytes)
cluster_size: 4096
Format specific information:
compat: 1.1
- compression type: zlib
+ compression type: COMPRESSION_TYPE
lazy refcounts: true
refcount bits: 16
corrupt: false
@@ -31,7 +31,7 @@ virtual size: 128 MiB (134217728 bytes)
cluster_size: 8192
Format specific information:
compat: 1.1
- compression type: zlib
+ compression type: COMPRESSION_TYPE
lazy refcounts: true
refcount bits: 16
corrupt: false
@@ -329,7 +329,7 @@ virtual size: 128 MiB (134217728 bytes)
cluster_size: 4096
Format specific information:
compat: 1.1
- compression type: zlib
+ compression type: COMPRESSION_TYPE
lazy refcounts: true
refcount bits: 16
corrupt: false
@@ -342,7 +342,7 @@ virtual size: 128 MiB (134217728 bytes)
cluster_size: 8192
Format specific information:
compat: 1.1
- compression type: zlib
+ compression type: COMPRESSION_TYPE
lazy refcounts: true
refcount bits: 16
corrupt: false
@@ -639,7 +639,7 @@ virtual size: 128 MiB (134217728 bytes)
cluster_size: 65536
Format specific information:
compat: 1.1
- compression type: zlib
+ compression type: COMPRESSION_TYPE
lazy refcounts: true
refcount bits: 16
corrupt: false
@@ -652,7 +652,7 @@ virtual size: 130 MiB (136314880 bytes)
cluster_size: 65536
Format specific information:
compat: 1.1
- compression type: zlib
+ compression type: COMPRESSION_TYPE
lazy refcounts: false
refcount bits: 16
corrupt: false
@@ -665,7 +665,7 @@ virtual size: 132 MiB (138412032 bytes)
cluster_size: 65536
Format specific information:
compat: 1.1
- compression type: zlib
+ compression type: COMPRESSION_TYPE
lazy refcounts: true
refcount bits: 16
corrupt: false
diff --git a/tests/qemu-iotests/198.out b/tests/qemu-iotests/198.out
index 3952708444799fede03c45cce0b7..805494916f102694a38729b8a217 100644
--- a/tests/qemu-iotests/198.out
+++ b/tests/qemu-iotests/198.out
@@ -36,7 +36,7 @@ image: json:{ /* filtered */ }
file format: IMGFMT
virtual size: 16 MiB (16777216 bytes)
Format specific information:
- compression type: zlib
+ compression type: COMPRESSION_TYPE
encrypt:
ivgen alg: plain64
hash alg: sha256
@@ -81,7 +81,7 @@ virtual size: 16 MiB (16777216 bytes)
backing file: TEST_DIR/t.IMGFMT.base
backing file format: IMGFMT
Format specific information:
- compression type: zlib
+ compression type: COMPRESSION_TYPE
encrypt:
ivgen alg: plain64
hash alg: sha256
diff --git a/tests/qemu-iotests/287 b/tests/qemu-iotests/287
index 5427ad5456aba89d04e573758679..6414640b2121d6511d7dcb11cda1 100755
--- a/tests/qemu-iotests/287
+++ b/tests/qemu-iotests/287
@@ -61,13 +61,13 @@ echo
echo "=== Testing compression type incompatible bit setting for zlib ==="
echo
_make_test_img -o compression_type=zlib 64M
-_qcow2_dump_header | grep incompatible_features
+_qcow2_dump_header --no-filter-compression | grep incompatible_features
echo
echo "=== Testing compression type incompatible bit setting for zstd ==="
echo
_make_test_img -o compression_type=zstd 64M
-_qcow2_dump_header | grep incompatible_features
+_qcow2_dump_header --no-filter-compression | grep incompatible_features
echo
echo "=== Testing zlib with incompatible bit set ==="
@@ -75,7 +75,7 @@ echo
_make_test_img -o compression_type=zlib 64M
$PYTHON qcow2.py "$TEST_IMG" set-feature-bit incompatible 3
# to make sure the bit was actually set
-_qcow2_dump_header | grep incompatible_features
+_qcow2_dump_header --no-filter-compression | grep incompatible_features
if $QEMU_IMG info "$TEST_IMG" >/dev/null 2>&1 ; then
echo "Error: The image opened successfully. The image must not be opened."
@@ -87,7 +87,7 @@ echo
_make_test_img -o compression_type=zstd 64M
$PYTHON qcow2.py "$TEST_IMG" set-header incompatible_features 0
# to make sure the bit was actually unset
-_qcow2_dump_header | grep incompatible_features
+_qcow2_dump_header --no-filter-compression | grep incompatible_features
if $QEMU_IMG info "$TEST_IMG" >/dev/null 2>&1 ; then
echo "Error: The image opened successfully. The image must not be opened."
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index 2b2b53946c65d698c71520a163a8..75cc241580d0289c70d7761e9758 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -247,6 +247,7 @@ _filter_img_info()
-e "/block_state_zero: \\(on\\|off\\)/d" \
-e "/log_size: [0-9]\\+/d" \
-e "s/iters: [0-9]\\+/iters: 1024/" \
+ -e 's/\(compression type: \)\(zlib\|zstd\)/\1COMPRESSION_TYPE/' \
-e "s/uuid: [-a-f0-9]\\+/uuid: 00000000-0000-0000-0000-000000000000/" | \
while IFS='' read -r line; do
if [[ $format_specific == 1 ]]; then
@@ -337,5 +338,12 @@ _filter_authz_check_tls()
$SED -e 's/TLS x509 authz check for .* is denied/TLS x509 authz check for DISTINGUISHED-NAME is denied/'
}
+_filter_qcow2_compression_type_bit()
+{
+ $SED -e 's/\(incompatible_features\s\+\)\[3\(, \)\?/\1[/' \
+ -e 's/\(incompatible_features.*\), 3\]/\1]/' \
+ -e 's/\(incompatible_features.*\), 3\(,.*\)/\1\2/'
+}
+
# make sure this script returns success
true
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index 5dea310ea08c2bca65a6e21cbb66..9885030b43c953bf073aa84a3df1 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -699,6 +699,7 @@ _img_info()
-e "s#$TEST_DIR#TEST_DIR#g" \
-e "s#$SOCK_DIR/fuse-#TEST_DIR/#g" \
-e "s#$IMGFMT#IMGFMT#g" \
+ -e 's/\(compression type: \)\(zlib\|zstd\)/\1COMPRESSION_TYPE/' \
-e "/^disk size:/ D" \
-e "/actual-size/ D" | \
while IFS='' read -r line; do
@@ -998,12 +999,23 @@ _require_one_device_of()
_qcow2_dump_header()
{
+ if [[ "$1" == "--no-filter-compression" ]]; then
+ local filter_compression=0
+ shift
+ else
+ local filter_compression=1
+ fi
+
img="$1"
if [ -z "$img" ]; then
img="$TEST_IMG"
fi
- $PYTHON qcow2.py "$img" dump-header
+ if [[ $filter_compression == 0 ]]; then
+ $PYTHON qcow2.py "$img" dump-header
+ else
+ $PYTHON qcow2.py "$img" dump-header | _filter_qcow2_compression_type_bit
+ fi
}
# make sure this script returns success

38
iotests-common.rc-introduce-_qcow2_dump_.patch
View File

@ -1,38 +0,0 @@
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Thu, 23 Dec 2021 17:01:38 +0100
Subject: iotests/common.rc: introduce _qcow2_dump_header helper
Git-commit: c5e627a6ecdccea64a1b600857ed671a83377847
We'll use it in tests instead of explicit qcow2.py. Then we are going
to add some filtering in _qcow2_dump_header.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20211223160144.1097696-14-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
tests/qemu-iotests/common.rc | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index d8582454de0aa4c2c9cfba0f0ce4..5dea310ea08c2bca65a6e21cbb66 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -996,5 +996,15 @@ _require_one_device_of()
_notrun "$* not available"
}
+_qcow2_dump_header()
+{
+ img="$1"
+ if [ -z "$img" ]; then
+ img="$TEST_IMG"
+ fi
+
+ $PYTHON qcow2.py "$img" dump-header
+}
+
# make sure this script returns success
true

98
iotests-declare-lack-of-support-for-comp.patch
View File

@ -1,98 +0,0 @@
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Thu, 23 Dec 2021 17:01:44 +0100
Subject: iotests: declare lack of support for compresion_type in IMGOPTS
Git-commit: e287a351db13f3f6670f4a3f2896f2dde47d07d1
compression_type can't be used if we want to create image with
compat=0.10. So, skip these tests, not many of them.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223160144.1097696-20-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
tests/qemu-iotests/031 | 5 +++--
tests/qemu-iotests/051 | 5 +++--
tests/qemu-iotests/061 | 6 +++++-
tests/qemu-iotests/112 | 3 ++-
tests/qemu-iotests/290 | 2 +-
5 files changed, 14 insertions(+), 7 deletions(-)
diff --git a/tests/qemu-iotests/031 b/tests/qemu-iotests/031
index 648112f7960e909e81fbc4c1be13..ee587b1606fd1b9c8ba1bf6a50b2 100755
--- a/tests/qemu-iotests/031
+++ b/tests/qemu-iotests/031
@@ -42,8 +42,9 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt qcow2
_supported_proto file fuse
# We want to test compat=0.10, which does not support external data
-# files or refcount widths other than 16
-_unsupported_imgopts data_file 'refcount_bits=\([^1]\|.\([^6]\|$\)\)'
+# files or refcount widths other than 16 or compression type
+_unsupported_imgopts data_file compression_type \
+ 'refcount_bits=\([^1]\|.\([^6]\|$\)\)'
CLUSTER_SIZE=65536
diff --git a/tests/qemu-iotests/051 b/tests/qemu-iotests/051
index e9042a621420332ac288060cbec2..f1a506518b981fe28cc498af43bb 100755
--- a/tests/qemu-iotests/051
+++ b/tests/qemu-iotests/051
@@ -41,8 +41,9 @@ _supported_fmt qcow2
_supported_proto file
# A compat=0.10 image is created in this test which does not support anything
# other than refcount_bits=16;
-# it also will not support an external data file
-_unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' data_file
+# it also will not support an external data file and compression type
+_unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' data_file \
+ compression_type
_require_drivers nbd
if [ "$QEMU_DEFAULT_MACHINE" = "pc" ]; then
diff --git a/tests/qemu-iotests/061 b/tests/qemu-iotests/061
index 70edf1a1635e1e5df529e130d314..513fbec14cc2bcfbd085b0f092e4 100755
--- a/tests/qemu-iotests/061
+++ b/tests/qemu-iotests/061
@@ -48,7 +48,11 @@ _supported_os Linux
# not work with it;
# we have explicit tests for various cluster sizes, the remaining tests
# require the default 64k cluster
-_unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' data_file cluster_size
+# we don't have explicit tests for zstd qcow2 compression type, as zstd may be
+# not compiled in. And we can't create compat images with comression type
+# extension
+_unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' data_file \
+ cluster_size compression_type
echo
echo "=== Testing version downgrade with zero expansion ==="
diff --git a/tests/qemu-iotests/112 b/tests/qemu-iotests/112
index 07ac74fb2c184c0580ebbedd70f9..53332129931fa67416893b3b2eaf 100755
--- a/tests/qemu-iotests/112
+++ b/tests/qemu-iotests/112
@@ -43,7 +43,8 @@ _supported_proto file fuse
# This test will set refcount_bits on its own which would conflict with the
# manual setting; compat will be overridden as well;
# and external data files do not work well with our refcount testing
-_unsupported_imgopts refcount_bits 'compat=0.10' data_file
+# also, compression type is not supported with compat=0.10 used in test
+_unsupported_imgopts refcount_bits 'compat=0.10' data_file compression_type
print_refcount_bits()
{
diff --git a/tests/qemu-iotests/290 b/tests/qemu-iotests/290
index ed80da2685e5ed094985e71c1e30..776b59de1bc8da4bbc67195231c4 100755
--- a/tests/qemu-iotests/290
+++ b/tests/qemu-iotests/290
@@ -41,7 +41,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt qcow2
_supported_proto file fuse
_supported_os Linux
-_unsupported_imgopts 'compat=0.10' refcount_bits data_file
+_unsupported_imgopts 'compat=0.10' refcount_bits data_file compression_type
echo
echo "### Test 'qemu-io -c discard' on a QCOW2 image without a backing file"

112
iotests-drop-qemu_img_verbose-helper.patch
View File

@ -1,112 +0,0 @@
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Thu, 23 Dec 2021 17:01:30 +0100
Subject: iotests: drop qemu_img_verbose() helper
Git-commit: 8f9e54ccfd047cbef09fe10fa75d59333052fb78
qemu_img_verbose() has a drawback of not going through generic
qemu_img_pipe_and_status(). qemu_img_verbose() is not very popular, so
update the only two users to qemu_img_log() and drop qemu_img_verbose()
at all.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223160144.1097696-6-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
tests/qemu-iotests/044 | 5 +++--
tests/qemu-iotests/044.out | 1 +
tests/qemu-iotests/209 | 7 ++++---
tests/qemu-iotests/209.out | 2 ++
tests/qemu-iotests/iotests.py | 8 --------
5 files changed, 10 insertions(+), 13 deletions(-)
diff --git a/tests/qemu-iotests/044 b/tests/qemu-iotests/044
index d696e6442ad5b595d16a61f5c0dd..a5ee9a7ded3bf1f734322db80689 100755
--- a/tests/qemu-iotests/044
+++ b/tests/qemu-iotests/044
@@ -24,7 +24,7 @@ import os
import qcow2
from qcow2 import QcowHeader
import iotests
-from iotests import qemu_img, qemu_img_verbose, qemu_io
+from iotests import qemu_img, qemu_img_log, qemu_io
import struct
import subprocess
import sys
@@ -112,10 +112,11 @@ class TestRefcountTableGrowth(iotests.QMPTestCase):
def test_grow_refcount_table(self):
qemu_io('-c', 'write 3800M 1M', test_img)
- qemu_img_verbose('check' , test_img)
+ qemu_img_log('check' , test_img)
pass
if __name__ == '__main__':
+ iotests.activate_logging()
iotests.main(supported_fmts=['qcow2'],
supported_protocols=['file'],
unsupported_imgopts=['refcount_bits'])
diff --git a/tests/qemu-iotests/044.out b/tests/qemu-iotests/044.out
index 703cf3dee186906c6b7b0104f142..ff663b17d71f96d1d9ade8b45bdd 100644
--- a/tests/qemu-iotests/044.out
+++ b/tests/qemu-iotests/044.out
@@ -1,6 +1,7 @@
No errors were found on the image.
7292415/33554432 = 21.73% allocated, 0.00% fragmented, 0.00% compressed clusters
Image end offset: 4296217088
+
.
----------------------------------------------------------------------
Ran 1 tests
diff --git a/tests/qemu-iotests/209 b/tests/qemu-iotests/209
index ff7efea11bc69d64ed3ba41c586d..f6ad08ec42a7055634adfcd263d1 100755
--- a/tests/qemu-iotests/209
+++ b/tests/qemu-iotests/209
@@ -20,8 +20,8 @@
#
import iotests
-from iotests import qemu_img_create, qemu_io, qemu_img_verbose, qemu_nbd, \
- file_path
+from iotests import qemu_img_create, qemu_io, qemu_img_log, qemu_nbd, \
+ file_path, log
iotests.script_initialize(supported_fmts=['qcow2'])
@@ -33,4 +33,5 @@ qemu_img_create('-f', iotests.imgfmt, disk, '1M')
qemu_io('-f', iotests.imgfmt, '-c', 'write 0 512K', disk)
qemu_nbd('-k', nbd_sock, '-x', 'exp', '-f', iotests.imgfmt, disk)
-qemu_img_verbose('map', '-f', 'raw', '--output=json', nbd_uri)
+qemu_img_log('map', '-f', 'raw', '--output=json', nbd_uri)
+log('done.') # avoid new line at the end of output file
diff --git a/tests/qemu-iotests/209.out b/tests/qemu-iotests/209.out
index f27be3fa7b0bf619ae5ff0a308fb..515906ac7a1962b78ef3787a4a7c 100644
--- a/tests/qemu-iotests/209.out
+++ b/tests/qemu-iotests/209.out
@@ -1,2 +1,4 @@
[{ "start": 0, "length": 524288, "depth": 0, "present": true, "zero": false, "data": true, "offset": 0},
{ "start": 524288, "length": 524288, "depth": 0, "present": true, "zero": true, "data": false, "offset": 524288}]
+
+done.
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index f0370b520651aa4ccc1cdb0cf28e..5a6bfcbb6adbb84d412e37966360 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -227,14 +227,6 @@ def qemu_img_measure(*args):
def qemu_img_check(*args):
return json.loads(qemu_img_pipe("check", "--output", "json", *args))
-def qemu_img_verbose(*args):
- '''Run qemu-img without suppressing its output and return the exit code'''
- exitcode = subprocess.call(qemu_img_args + list(args))
- if exitcode < 0:
- sys.stderr.write('qemu-img received signal %i: %s\n'
- % (-exitcode, ' '.join(qemu_img_args + list(args))))
- return exitcode
-
def qemu_img_pipe(*args: str) -> str:
'''Run qemu-img and return its output'''
return qemu_img_pipe_and_status(*args)[0]

408
iotests-massive-use-_qcow2_dump_header.patch
View File

@ -1,408 +0,0 @@
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Thu, 23 Dec 2021 17:01:39 +0100
Subject: iotests: massive use _qcow2_dump_header
Git-commit: 984d7a52d5ca33a79e09f2617fe43e368dce4068
We are going to add filtering in _qcow2_dump_header and want all tests
use it.
The patch is generated by commands:
cd tests/qemu-iotests
sed -ie 's/$PYTHON qcow2.py "$TEST_IMG" dump-header\($\| \)/_qcow2_dump_header\1/' ??? tests/*
(the difficulty is to avoid converting dump-header-exts)
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20211223160144.1097696-15-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
(cherry picked from commit 984d7a52d5ca33a79e09f2617fe43e368dce4068)
Signed-off-by: Li Zhang <lizhang@suse.de>
---
tests/qemu-iotests/031 | 6 +++---
tests/qemu-iotests/036 | 6 +++---
tests/qemu-iotests/039 | 20 ++++++++++----------
tests/qemu-iotests/060 | 20 ++++++++++----------
tests/qemu-iotests/061 | 36 ++++++++++++++++++------------------
tests/qemu-iotests/137 | 2 +-
tests/qemu-iotests/287 | 8 ++++----
7 files changed, 49 insertions(+), 49 deletions(-)
diff --git a/tests/qemu-iotests/031 b/tests/qemu-iotests/031
index 58b57a0ef2c0973b6f595102802d..648112f7960e909e81fbc4c1be13 100755
--- a/tests/qemu-iotests/031
+++ b/tests/qemu-iotests/031
@@ -58,21 +58,21 @@ for compat in "compat=0.10" "compat=1.1"; do
echo
_make_test_img -o $compat 64M
$PYTHON qcow2.py "$TEST_IMG" add-header-ext 0x12345678 "This is a test header extension"
- $PYTHON qcow2.py "$TEST_IMG" dump-header
+ _qcow2_dump_header
_check_test_img
echo
echo === Rewrite header with no backing file ===
echo
$QEMU_IMG rebase -u -b "" "$TEST_IMG"
- $PYTHON qcow2.py "$TEST_IMG" dump-header
+ _qcow2_dump_header
_check_test_img
echo
echo === Add a backing file and format ===
echo
$QEMU_IMG rebase -u -b "/some/backing/file/path" -F host_device "$TEST_IMG"
- $PYTHON qcow2.py "$TEST_IMG" dump-header
+ _qcow2_dump_header
done
# success, all done
diff --git a/tests/qemu-iotests/036 b/tests/qemu-iotests/036
index 5e567012a8203e38b609fd863736..f703605e44412b70e3f2aa9ea33f 100755
--- a/tests/qemu-iotests/036
+++ b/tests/qemu-iotests/036
@@ -58,7 +58,7 @@ $PYTHON qcow2.py "$TEST_IMG" set-feature-bit incompatible 63
# Without feature table
$PYTHON qcow2.py "$TEST_IMG" del-header-ext 0x6803f857
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep features
+_qcow2_dump_header | grep features
$PYTHON qcow2.py "$TEST_IMG" dump-header-exts
_img_info
@@ -107,7 +107,7 @@ echo === Create image with unknown autoclear feature bit ===
echo
_make_test_img 64M
$PYTHON qcow2.py "$TEST_IMG" set-feature-bit autoclear 63
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep features
+_qcow2_dump_header | grep features
$PYTHON qcow2.py "$TEST_IMG" dump-header-exts
echo
@@ -115,7 +115,7 @@ echo === Repair image ===
echo
_check_test_img -r all
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep features
+_qcow2_dump_header | grep features
$PYTHON qcow2.py "$TEST_IMG" dump-header-exts
# success, all done
diff --git a/tests/qemu-iotests/039 b/tests/qemu-iotests/039
index 12b2c7fa7b87d87db9c5f5688dfe..8e783a8380666f26d237c7233602 100755
--- a/tests/qemu-iotests/039
+++ b/tests/qemu-iotests/039
@@ -59,7 +59,7 @@ _make_test_img -o "compat=1.1,lazy_refcounts=on" $size
$QEMU_IO -c "write -P 0x5a 0 512" "$TEST_IMG" | _filter_qemu_io
# The dirty bit must not be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
_check_test_img
echo
@@ -73,7 +73,7 @@ $QEMU_IO -c "write -P 0x5a 0 512" \
| _filter_qemu_io
# The dirty bit must be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
_check_test_img
echo
@@ -82,7 +82,7 @@ echo "== Read-only access must still work =="
$QEMU_IO -r -c "read -P 0x5a 0 512" "$TEST_IMG" | _filter_qemu_io
# The dirty bit must be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
echo
echo "== Repairing the image file must succeed =="
@@ -90,7 +90,7 @@ echo "== Repairing the image file must succeed =="
_check_test_img -r all
# The dirty bit must not be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
echo
echo "== Data should still be accessible after repair =="
@@ -108,12 +108,12 @@ $QEMU_IO -c "write -P 0x5a 0 512" \
| _filter_qemu_io
# The dirty bit must be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
$QEMU_IO -c "write 0 512" "$TEST_IMG" | _filter_qemu_io
# The dirty bit must not be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
echo
echo "== Creating an image file with lazy_refcounts=off =="
@@ -126,7 +126,7 @@ $QEMU_IO -c "write -P 0x5a 0 512" \
| _filter_qemu_io
# The dirty bit must not be set since lazy_refcounts=off
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
_check_test_img
echo
@@ -141,7 +141,7 @@ $QEMU_IO -c "write 0 512" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG commit "$TEST_IMG"
# The dirty bit must not be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
$PYTHON qcow2.py "$TEST_IMG".base dump-header | grep incompatible_features
_check_test_img
@@ -159,7 +159,7 @@ $QEMU_IO -c "reopen -o lazy-refcounts=on" \
| _filter_qemu_io
# The dirty bit must be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
_check_test_img
_make_test_img -o "compat=1.1,lazy_refcounts=on" $size
@@ -171,7 +171,7 @@ $QEMU_IO -c "reopen -o lazy-refcounts=off" \
| _filter_qemu_io
# The dirty bit must not be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
_check_test_img
diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060
index db26c6b246cda94e9258c51b0b6c..d1e3204d4ea42002263486eebafd 100755
--- a/tests/qemu-iotests/060
+++ b/tests/qemu-iotests/060
@@ -80,13 +80,13 @@ poke_file "$TEST_IMG" "$l1_offset" "\x80\x00\x00\x00\x00\x03\x00\x00"
_check_test_img
# The corrupt bit should not be set anyway
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
# Try to write something, thereby forcing the corrupt bit to be set
$QEMU_IO -c "$OPEN_RW" -c "write -P 0x2a 0 512" | _filter_qemu_io
# The corrupt bit must now be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
# This information should be available through qemu-img info
_img_info --format-specific
@@ -114,19 +114,19 @@ poke_file "$TEST_IMG" "$(($rb_offset+8))" "\x00\x01"
# Redirect new data cluster onto refcount block
poke_file "$TEST_IMG" "$l2_offset" "\x80\x00\x00\x00\x00\x02\x00\x00"
_check_test_img
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
$QEMU_IO -c "$OPEN_RW" -c "write -P 0x2a 0 512" | _filter_qemu_io
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
# Try to fix it
_check_test_img -r all
# The corrupt bit should be cleared
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
# Look if it's really really fixed
$QEMU_IO -c "$OPEN_RW" -c "write -P 0x2a 0 512" | _filter_qemu_io
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
echo
echo "=== Testing cluster data reference into inactive L2 table ==="
@@ -139,13 +139,13 @@ $QEMU_IO -c "$OPEN_RW" -c "write -P 2 0 512" | _filter_qemu_io
poke_file "$TEST_IMG" "$l2_offset_after_snapshot" \
"\x80\x00\x00\x00\x00\x04\x00\x00"
_check_test_img
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
$QEMU_IO -c "$OPEN_RW" -c "write -P 3 0 512" | _filter_qemu_io
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
_check_test_img -r all
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
$QEMU_IO -c "$OPEN_RW" -c "write -P 4 0 512" | _filter_qemu_io
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
# Check data
$QEMU_IO -c "$OPEN_RO" -c "read -P 4 0 512" | _filter_qemu_io
diff --git a/tests/qemu-iotests/061 b/tests/qemu-iotests/061
index 9507c223bda41067797dd19be095..70edf1a1635e1e5df529e130d314 100755
--- a/tests/qemu-iotests/061
+++ b/tests/qemu-iotests/061
@@ -55,9 +55,9 @@ echo "=== Testing version downgrade with zero expansion ==="
echo
_make_test_img -o "compat=1.1,lazy_refcounts=on" 64M
$QEMU_IO -c "write -z 0 128k" "$TEST_IMG" | _filter_qemu_io
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
$QEMU_IMG amend -o "compat=0.10" "$TEST_IMG"
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
$QEMU_IO -c "read -P 0 0 128k" "$TEST_IMG" | _filter_qemu_io
_check_test_img
@@ -68,10 +68,10 @@ _make_test_img -o "compat=1.1,lazy_refcounts=on" 64M
$QEMU_IO -c "write -z 0 128k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "write -z 32M 128k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c map "$TEST_IMG" | _filter_qemu_io
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
$QEMU_IMG amend -o "compat=0.10" --image-opts \
driver=qcow2,file.filename=$TEST_IMG,l2-cache-entry-size=4096
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
$QEMU_IO -c "read -P 0 0 128k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 32M 128k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c map "$TEST_IMG" | _filter_qemu_io
@@ -84,9 +84,9 @@ _make_test_img -o "compat=1.1,lazy_refcounts=on" 64M
_NO_VALGRIND \
$QEMU_IO -c "write -P 0x2a 0 128k" -c flush \
-c "sigraise $(kill -l KILL)" "$TEST_IMG" 2>&1 | _filter_qemu_io
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
$QEMU_IMG amend -o "compat=0.10" "$TEST_IMG"
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
$QEMU_IO -c "read -P 0x2a 0 128k" "$TEST_IMG" | _filter_qemu_io
_check_test_img
@@ -96,9 +96,9 @@ echo
_make_test_img -o "compat=1.1" 64M
$PYTHON qcow2.py "$TEST_IMG" set-feature-bit compatible 42
$PYTHON qcow2.py "$TEST_IMG" set-feature-bit autoclear 42
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
$QEMU_IMG amend -o "compat=0.10" "$TEST_IMG"
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
_check_test_img
echo
@@ -106,9 +106,9 @@ echo "=== Testing version upgrade and resize ==="
echo
_make_test_img -o "compat=0.10" 64M
$QEMU_IO -c "write -P 0x2a 42M 64k" "$TEST_IMG" | _filter_qemu_io
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
$QEMU_IMG amend -o "compat=1.1,lazy_refcounts=on,size=128M" "$TEST_IMG"
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
$QEMU_IO -c "read -P 0x2a 42M 64k" "$TEST_IMG" | _filter_qemu_io
_check_test_img
@@ -120,29 +120,29 @@ $QEMU_IO -c "write -P 0x2a 24M 64k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG snapshot -c foo "$TEST_IMG"
$QEMU_IMG resize "$TEST_IMG" 64M &&
echo "unexpected pass"
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep '^\(version\|size\|nb_snap\)'
+_qcow2_dump_header | grep '^\(version\|size\|nb_snap\)'
$QEMU_IMG amend -o "compat=1.1,size=128M" "$TEST_IMG" ||
echo "unexpected fail"
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep '^\(version\|size\|nb_snap\)'
+_qcow2_dump_header | grep '^\(version\|size\|nb_snap\)'
$QEMU_IMG snapshot -c bar "$TEST_IMG"
$QEMU_IMG resize --shrink "$TEST_IMG" 64M ||
echo "unexpected fail"
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep '^\(version\|size\|nb_snap\)'
+_qcow2_dump_header | grep '^\(version\|size\|nb_snap\)'
$QEMU_IMG amend -o "compat=0.10,size=32M" "$TEST_IMG" &&
echo "unexpected pass"
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep '^\(version\|size\|nb_snap\)'
+_qcow2_dump_header | grep '^\(version\|size\|nb_snap\)'
$QEMU_IMG snapshot -a bar "$TEST_IMG" ||
echo "unexpected fail"
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep '^\(version\|size\|nb_snap\)'
+_qcow2_dump_header | grep '^\(version\|size\|nb_snap\)'
$QEMU_IMG snapshot -d bar "$TEST_IMG"
$QEMU_IMG amend -o "compat=0.10,size=32M" "$TEST_IMG" ||
echo "unexpected fail"
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep '^\(version\|size\|nb_snap\)'
+_qcow2_dump_header | grep '^\(version\|size\|nb_snap\)'
_check_test_img
@@ -154,9 +154,9 @@ _make_test_img -o "compat=1.1,lazy_refcounts=on" 64M
_NO_VALGRIND \
$QEMU_IO -c "write -P 0x2a 0 128k" -c flush \
-c "sigraise $(kill -l KILL)" "$TEST_IMG" 2>&1 | _filter_qemu_io
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
$QEMU_IMG amend -o "lazy_refcounts=off" "$TEST_IMG"
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
$QEMU_IO -c "read -P 0x2a 0 128k" "$TEST_IMG" | _filter_qemu_io
_check_test_img
diff --git a/tests/qemu-iotests/137 b/tests/qemu-iotests/137
index 4680d5df3d0fe8be4041f4df3833..52ee135184011e8e1655efd063f9 100755
--- a/tests/qemu-iotests/137
+++ b/tests/qemu-iotests/137
@@ -140,7 +140,7 @@ $QEMU_IO \
# The dirty bit must not be set
# (Filter the external data file bit)
-if $PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features \
+if _qcow2_dump_header | grep incompatible_features \
| grep -q '\<0\>'
then
echo 'ERROR: Dirty bit set'
diff --git a/tests/qemu-iotests/287 b/tests/qemu-iotests/287
index 2d5334e8bfb72dc88a3ddd958994..5427ad5456aba89d04e573758679 100755
--- a/tests/qemu-iotests/287
+++ b/tests/qemu-iotests/287
@@ -61,13 +61,13 @@ echo
echo "=== Testing compression type incompatible bit setting for zlib ==="
echo
_make_test_img -o compression_type=zlib 64M
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
echo
echo "=== Testing compression type incompatible bit setting for zstd ==="
echo
_make_test_img -o compression_type=zstd 64M
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
echo
echo "=== Testing zlib with incompatible bit set ==="
@@ -75,7 +75,7 @@ echo
_make_test_img -o compression_type=zlib 64M
$PYTHON qcow2.py "$TEST_IMG" set-feature-bit incompatible 3
# to make sure the bit was actually set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
if $QEMU_IMG info "$TEST_IMG" >/dev/null 2>&1 ; then
echo "Error: The image opened successfully. The image must not be opened."
@@ -87,7 +87,7 @@ echo
_make_test_img -o compression_type=zstd 64M
$PYTHON qcow2.py "$TEST_IMG" set-header incompatible_features 0
# to make sure the bit was actually unset
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
if $QEMU_IMG info "$TEST_IMG" >/dev/null 2>&1 ; then
echo "Error: The image opened successfully. The image must not be opened."

222
iotests-specify-some-unsupported_imgopts.patch
View File

@ -1,222 +0,0 @@
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Thu, 23 Dec 2021 17:01:28 +0100
Subject: iotests: specify some unsupported_imgopts for python iotests
Git-commit: b30b8077243ea5dc93a540eedfecee3c74b19fa2
We are going to support IMGOPTS for python iotests. Still some iotests
will not work with common IMGOPTS used with bash iotests like
specifying refcount_bits and compat qcow2 options. So we
should define corresponding unsupported_imgopts for now.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20211223160144.1097696-4-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
tests/qemu-iotests/044 | 3 ++-
tests/qemu-iotests/065 | 3 ++-
tests/qemu-iotests/163 | 3 ++-
tests/qemu-iotests/165 | 3 ++-
tests/qemu-iotests/196 | 3 ++-
tests/qemu-iotests/242 | 3 ++-
tests/qemu-iotests/246 | 3 ++-
tests/qemu-iotests/254 | 3 ++-
tests/qemu-iotests/260 | 3 ++-
tests/qemu-iotests/274 | 3 ++-
tests/qemu-iotests/281 | 3 ++-
tests/qemu-iotests/303 | 3 ++-
tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test | 3 ++-
tests/qemu-iotests/tests/migrate-bitmaps-test | 3 ++-
tests/qemu-iotests/tests/remove-bitmap-from-backing | 3 ++-
15 files changed, 30 insertions(+), 15 deletions(-)
diff --git a/tests/qemu-iotests/044 b/tests/qemu-iotests/044
index 64b18eb7c89f49c8fc7192dbb0e8..d696e6442ad5b595d16a61f5c0dd 100755
--- a/tests/qemu-iotests/044
+++ b/tests/qemu-iotests/044
@@ -117,4 +117,5 @@ class TestRefcountTableGrowth(iotests.QMPTestCase):
if __name__ == '__main__':
iotests.main(supported_fmts=['qcow2'],
- supported_protocols=['file'])
+ supported_protocols=['file'],
+ unsupported_imgopts=['refcount_bits'])
diff --git a/tests/qemu-iotests/065 b/tests/qemu-iotests/065
index 3c2ca27627f0d6343ba04c857b38..dc7716275f0d5e8b6aeb572244fb 100755
--- a/tests/qemu-iotests/065
+++ b/tests/qemu-iotests/065
@@ -139,4 +139,5 @@ TestQMP = None
if __name__ == '__main__':
iotests.main(supported_fmts=['qcow2'],
- supported_protocols=['file'])
+ supported_protocols=['file'],
+ unsupported_imgopts=['refcount_bits'])
diff --git a/tests/qemu-iotests/163 b/tests/qemu-iotests/163
index dedce8ef4322d3cec0303049b280..b8bfc95358e4652c199655030c8a 100755
--- a/tests/qemu-iotests/163
+++ b/tests/qemu-iotests/163
@@ -169,4 +169,5 @@ ShrinkBaseClass = None
if __name__ == '__main__':
iotests.main(supported_fmts=['raw', 'qcow2'],
- supported_protocols=['file'])
+ supported_protocols=['file'],
+ unsupported_imgopts=['compat'])
diff --git a/tests/qemu-iotests/165 b/tests/qemu-iotests/165
index ce499946b842e71f45c7b24ca162..e3ef28e2ee84277ff7bbe65d5fff 100755
--- a/tests/qemu-iotests/165
+++ b/tests/qemu-iotests/165
@@ -157,4 +157,5 @@ class TestPersistentDirtyBitmap(iotests.QMPTestCase):
if __name__ == '__main__':
iotests.main(supported_fmts=['qcow2'],
- supported_protocols=['file'])
+ supported_protocols=['file'],
+ unsupported_imgopts=['compat'])
diff --git a/tests/qemu-iotests/196 b/tests/qemu-iotests/196
index 2451515094f2e582dc8dbe838e60..76509a5ad1c541f572ab9c918046 100755
--- a/tests/qemu-iotests/196
+++ b/tests/qemu-iotests/196
@@ -65,4 +65,5 @@ class TestInvalidateAutoclear(iotests.QMPTestCase):
if __name__ == '__main__':
iotests.main(supported_fmts=['qcow2'],
- supported_protocols=['file'])
+ supported_protocols=['file'],
+ unsupported_imgopts=['compat'])
diff --git a/tests/qemu-iotests/242 b/tests/qemu-iotests/242
index a9b27668c2757711fc3f21db87d4..96a30152b04a8fba7eed2514a48c 100755
--- a/tests/qemu-iotests/242
+++ b/tests/qemu-iotests/242
@@ -26,7 +26,8 @@ from iotests import qemu_img_create, qemu_io, qemu_img_pipe, \
file_path, img_info_log, log, filter_qemu_io
iotests.script_initialize(supported_fmts=['qcow2'],
- supported_protocols=['file'])
+ supported_protocols=['file'],
+ unsupported_imgopts=['refcount_bits', 'compat'])
disk = file_path('disk')
chunk = 256 * 1024
diff --git a/tests/qemu-iotests/246 b/tests/qemu-iotests/246
index 5932a0e8a97bdd5975a4bc0813f9..b009a78397752bdd60bf2ec34af5 100755
--- a/tests/qemu-iotests/246
+++ b/tests/qemu-iotests/246
@@ -23,7 +23,8 @@
import iotests
from iotests import log
-iotests.script_initialize(supported_fmts=['qcow2'])
+iotests.script_initialize(supported_fmts=['qcow2'],
+ unsupported_imgopts=['compat'])
size = 64 * 1024 * 1024 * 1024
gran_small = 32 * 1024
gran_large = 128 * 1024
diff --git a/tests/qemu-iotests/254 b/tests/qemu-iotests/254
index 108bf5f894c6b3328c144a5de1ee..7ea098818cf0a39c0a624989cdc8 100755
--- a/tests/qemu-iotests/254
+++ b/tests/qemu-iotests/254
@@ -22,7 +22,8 @@
import iotests
from iotests import qemu_img_create, file_path, log
-iotests.script_initialize(supported_fmts=['qcow2'])
+iotests.script_initialize(supported_fmts=['qcow2'],
+ unsupported_imgopts=['compat'])
disk, top = file_path('disk', 'top')
size = 1024 * 1024
diff --git a/tests/qemu-iotests/260 b/tests/qemu-iotests/260
index 2ec64a9b9952cd26ac7c795fca36..c2133f998010bef3111abb98f0be 100755
--- a/tests/qemu-iotests/260
+++ b/tests/qemu-iotests/260
@@ -23,7 +23,8 @@ import iotests
from iotests import qemu_img_create, file_path, log, filter_qmp_event
iotests.script_initialize(
- supported_fmts=['qcow2']
+ supported_fmts=['qcow2'],
+ unsupported_imgopts=['compat']
)
base, top = file_path('base', 'top')
diff --git a/tests/qemu-iotests/274 b/tests/qemu-iotests/274
index caab008e0737ba5d88e2ae46bc11..080a90f10f7af42c09e380b5953b 100755
--- a/tests/qemu-iotests/274
+++ b/tests/qemu-iotests/274
@@ -23,7 +23,8 @@
import iotests
iotests.script_initialize(supported_fmts=['qcow2'],
- supported_platforms=['linux'])
+ supported_platforms=['linux'],
+ unsupported_imgopts=['refcount_bits', 'compat'])
size_short = 1 * 1024 * 1024
size_long = 2 * 1024 * 1024
diff --git a/tests/qemu-iotests/281 b/tests/qemu-iotests/281
index 956698083f03a7d80f2076cc18c1..318e33393919f2b89295700bbf05 100755
--- a/tests/qemu-iotests/281
+++ b/tests/qemu-iotests/281
@@ -245,4 +245,5 @@ class TestBlockdevBackupAbort(iotests.QMPTestCase):
if __name__ == '__main__':
iotests.main(supported_fmts=['qcow2'],
- supported_protocols=['file'])
+ supported_protocols=['file'],
+ unsupported_imgopts=['compat'])
diff --git a/tests/qemu-iotests/303 b/tests/qemu-iotests/303
index 425544c064d247af86925696bf6d..475cb5428db4f047b7db63776ca0 100755
--- a/tests/qemu-iotests/303
+++ b/tests/qemu-iotests/303
@@ -23,7 +23,8 @@ import iotests
import subprocess
from iotests import qemu_img_create, qemu_io, file_path, log, filter_qemu_io
-iotests.script_initialize(supported_fmts=['qcow2'])
+iotests.script_initialize(supported_fmts=['qcow2'],
+ unsupported_imgopts=['refcount_bits', 'compat'])
disk = file_path('disk')
chunk = 1024 * 1024
diff --git a/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test b/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
index 00ebb5c2516fb7a6b9ee5c7cceaa..fc9c4b4ef411dc93e1abba0e58d2 100755
--- a/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
+++ b/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
@@ -272,4 +272,5 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCase):
if __name__ == '__main__':
- iotests.main(supported_fmts=['qcow2'])
+ iotests.main(supported_fmts=['qcow2'],
+ unsupported_imgopts=['compat'])
diff --git a/tests/qemu-iotests/tests/migrate-bitmaps-test b/tests/qemu-iotests/tests/migrate-bitmaps-test
index c23df3d75c7fd0efc5a4c79aa1f2..59f33575805c79a2ea24157fdb03 100755
--- a/tests/qemu-iotests/tests/migrate-bitmaps-test
+++ b/tests/qemu-iotests/tests/migrate-bitmaps-test
@@ -307,7 +307,8 @@ def main() -> None:
iotests.main(
supported_fmts=['qcow2'],
- supported_protocols=['file']
+ supported_protocols=['file'],
+ unsupported_imgopts=['compat']
)
diff --git a/tests/qemu-iotests/tests/remove-bitmap-from-backing b/tests/qemu-iotests/tests/remove-bitmap-from-backing
index 8d48fc0f3ce158182733d744b365..3c397b08ea439d6d4c9da76374c0 100755
--- a/tests/qemu-iotests/tests/remove-bitmap-from-backing
+++ b/tests/qemu-iotests/tests/remove-bitmap-from-backing
@@ -21,7 +21,8 @@
import iotests
from iotests import log, qemu_img_create, qemu_img, qemu_img_pipe
-iotests.script_initialize(supported_fmts=['qcow2'])
+iotests.script_initialize(supported_fmts=['qcow2'],
+ unsupported_imgopts=['compat'])
top, base = iotests.file_path('top', 'base')
size = '1M'

447
iotests.py-filter-out-successful-output-.patch
View File

@ -1,447 +0,0 @@
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Thu, 23 Dec 2021 17:01:34 +0100
Subject: iotests.py: filter out successful output of qemu-img create
Git-commit: e877bba308c38033f73f444395b3cdcde48a3393
The only "feature" of this "Formatting ..." line is that we have to
update it every time we add new option. Let's drop it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223160144.1097696-10-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
tests/qemu-iotests/149.out | 21 ---------------------
tests/qemu-iotests/237.out | 3 ---
tests/qemu-iotests/255.out | 4 ----
tests/qemu-iotests/274.out | 29 -----------------------------
tests/qemu-iotests/280.out | 1 -
tests/qemu-iotests/296.out | 10 +++-------
tests/qemu-iotests/iotests.py | 10 ++++++++--
7 files changed, 11 insertions(+), 67 deletions(-)
diff --git a/tests/qemu-iotests/149.out b/tests/qemu-iotests/149.out
index 6877ab6c4a4580d31d5565baad49..ab879596ce2f8811e236428afca5 100644
--- a/tests/qemu-iotests/149.out
+++ b/tests/qemu-iotests/149.out
@@ -61,7 +61,6 @@ unlink TEST_DIR/luks-aes-256-xts-plain64-sha1.img
# ================= qemu-img aes-256-xts-plain64-sha1 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-256-xts-plain64-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha1.img qiotest-145-aes-256-xts-plain64-sha1
@@ -181,7 +180,6 @@ unlink TEST_DIR/luks-twofish-256-xts-plain64-sha1.img
# ================= qemu-img twofish-256-xts-plain64-sha1 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=twofish-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-twofish-256-xts-plain64-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-twofish-256-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=twofish-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-twofish-256-xts-plain64-sha1.img qiotest-145-twofish-256-xts-plain64-sha1
@@ -301,7 +299,6 @@ unlink TEST_DIR/luks-serpent-256-xts-plain64-sha1.img
# ================= qemu-img serpent-256-xts-plain64-sha1 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=serpent-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-serpent-256-xts-plain64-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-serpent-256-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=serpent-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-256-xts-plain64-sha1.img qiotest-145-serpent-256-xts-plain64-sha1
@@ -421,7 +418,6 @@ unlink TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img
# ================= qemu-img cast5-128-cbc-plain64-sha1 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=cast5-128,cipher-mode=cbc,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=cast5-128 cipher-mode=cbc ivgen-alg=plain64 hash-alg=sha1 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img qiotest-145-cast5-128-cbc-plain64-sha1
@@ -542,7 +538,6 @@ unlink TEST_DIR/luks-aes-256-cbc-plain-sha1.img
# ================= qemu-img aes-256-cbc-plain-sha1 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=plain,hash-alg=sha1 TEST_DIR/luks-aes-256-cbc-plain-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-cbc-plain-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=plain hash-alg=sha1 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain-sha1.img qiotest-145-aes-256-cbc-plain-sha1
@@ -662,7 +657,6 @@ unlink TEST_DIR/luks-aes-256-cbc-plain64-sha1.img
# ================= qemu-img aes-256-cbc-plain64-sha1 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-256-cbc-plain64-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-cbc-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=plain64 hash-alg=sha1 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain64-sha1.img qiotest-145-aes-256-cbc-plain64-sha1
@@ -782,7 +776,6 @@ unlink TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img
# ================= qemu-img aes-256-cbc-essiv-sha256-sha1 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=essiv,hash-alg=sha1,ivgen-hash-alg=sha256 TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=essiv ivgen-hash-alg=sha256 hash-alg=sha1 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img qiotest-145-aes-256-cbc-essiv-sha256-sha1
@@ -902,7 +895,6 @@ unlink TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img
# ================= qemu-img aes-256-xts-essiv-sha256-sha1 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=essiv,hash-alg=sha1,ivgen-hash-alg=sha256 TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=essiv ivgen-hash-alg=sha256 hash-alg=sha1 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img qiotest-145-aes-256-xts-essiv-sha256-sha1
@@ -1022,7 +1014,6 @@ unlink TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img
# ================= qemu-img aes-128-xts-plain64-sha256-sha1 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-128,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-128 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img qiotest-145-aes-128-xts-plain64-sha256-sha1
@@ -1142,7 +1133,6 @@ unlink TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img
# ================= qemu-img aes-192-xts-plain64-sha256-sha1 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-192,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-192 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img qiotest-145-aes-192-xts-plain64-sha256-sha1
@@ -1262,7 +1252,6 @@ unlink TEST_DIR/luks-twofish-128-xts-plain64-sha1.img
# ================= qemu-img twofish-128-xts-plain64-sha1 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=twofish-128,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-twofish-128-xts-plain64-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-twofish-128-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=twofish-128 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-twofish-128-xts-plain64-sha1.img qiotest-145-twofish-128-xts-plain64-sha1
@@ -1383,7 +1372,6 @@ unlink TEST_DIR/luks-serpent-128-xts-plain64-sha1.img
# ================= qemu-img serpent-128-xts-plain64-sha1 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=serpent-128,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-serpent-128-xts-plain64-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-serpent-128-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=serpent-128 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-128-xts-plain64-sha1.img qiotest-145-serpent-128-xts-plain64-sha1
@@ -1503,7 +1491,6 @@ unlink TEST_DIR/luks-serpent-192-xts-plain64-sha1.img
# ================= qemu-img serpent-192-xts-plain64-sha1 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=serpent-192,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-serpent-192-xts-plain64-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-serpent-192-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=serpent-192 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-192-xts-plain64-sha1.img qiotest-145-serpent-192-xts-plain64-sha1
@@ -1625,7 +1612,6 @@ unlink TEST_DIR/luks-aes-256-xts-plain64-sha224.img
# ================= qemu-img aes-256-xts-plain64-sha224 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha224 TEST_DIR/luks-aes-256-xts-plain64-sha224.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha224.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha224 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha224.img qiotest-145-aes-256-xts-plain64-sha224
@@ -1745,7 +1731,6 @@ unlink TEST_DIR/luks-aes-256-xts-plain64-sha256.img
# ================= qemu-img aes-256-xts-plain64-sha256 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha256 TEST_DIR/luks-aes-256-xts-plain64-sha256.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha256.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha256 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha256.img qiotest-145-aes-256-xts-plain64-sha256
@@ -1865,7 +1850,6 @@ unlink TEST_DIR/luks-aes-256-xts-plain64-sha384.img
# ================= qemu-img aes-256-xts-plain64-sha384 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha384 TEST_DIR/luks-aes-256-xts-plain64-sha384.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha384.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha384 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha384.img qiotest-145-aes-256-xts-plain64-sha384
@@ -1985,7 +1969,6 @@ unlink TEST_DIR/luks-aes-256-xts-plain64-sha512.img
# ================= qemu-img aes-256-xts-plain64-sha512 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha512 TEST_DIR/luks-aes-256-xts-plain64-sha512.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha512.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha512 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha512.img qiotest-145-aes-256-xts-plain64-sha512
@@ -2105,7 +2088,6 @@ unlink TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img
# ================= qemu-img aes-256-xts-plain64-ripemd160 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=ripemd160 TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=ripemd160 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img qiotest-145-aes-256-xts-plain64-ripemd160
@@ -2299,7 +2281,6 @@ unlink TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img
# ================= qemu-img aes-256-xts-plain-sha1-pwallslots =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=c2xvdDE=,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain,hash-alg=sha1 TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain hash-alg=sha1 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img qiotest-145-aes-256-xts-plain-sha1-pwallslots
@@ -2419,7 +2400,6 @@ unlink TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img
# ================= qemu-img aes-256-cbc-essiv-auto-sha1 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=essiv,hash-alg=sha1 TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=essiv hash-alg=sha1 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img qiotest-145-aes-256-cbc-essiv-auto-sha1
@@ -2539,7 +2519,6 @@ unlink TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img
# ================= qemu-img aes-256-cbc-plain64-sha256-sha1 =================
# Create image
qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=plain64,hash-alg=sha1,ivgen-hash-alg=sha256 TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=plain64 ivgen-hash-alg=sha256 hash-alg=sha1 iter-time=10
# Open dev
sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img qiotest-145-aes-256-cbc-plain64-sha256-sha1
diff --git a/tests/qemu-iotests/237.out b/tests/qemu-iotests/237.out
index 2f09ff5512cf7e586280fe37c571..aeb972449289e73ab955b200c2b3 100644
--- a/tests/qemu-iotests/237.out
+++ b/tests/qemu-iotests/237.out
@@ -129,11 +129,8 @@ Job failed: Cannot find device='this doesn't exist' nor node-name='this doesn't
=== Other subformats ===
-Formatting 'TEST_DIR/PID-t.vmdk.1', fmt=vmdk size=0 compat6=off hwversion=undefined
-Formatting 'TEST_DIR/PID-t.vmdk.2', fmt=vmdk size=0 compat6=off hwversion=undefined
-Formatting 'TEST_DIR/PID-t.vmdk.3', fmt=vmdk size=0 compat6=off hwversion=undefined
== Missing extent ==
diff --git a/tests/qemu-iotests/255.out b/tests/qemu-iotests/255.out
index 33b7f22de3f2eb7ce724c2285a71..11a05a5213e6a66e1ead1fa1abc2 100644
--- a/tests/qemu-iotests/255.out
+++ b/tests/qemu-iotests/255.out
@@ -3,9 +3,7 @@ Finishing a commit job with background reads
=== Create backing chain and start VM ===
-Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
=== Start background read requests ===
@@ -23,9 +21,7 @@ Closing the VM while a job is being cancelled
=== Create images and start VM ===
-Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
wrote 1048576/1048576 bytes at offset 0
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
diff --git a/tests/qemu-iotests/274.out b/tests/qemu-iotests/274.out
index 16a95a48508b5c56ba146063a604..1d2928e14daac9aed9df887f566c 100644
--- a/tests/qemu-iotests/274.out
+++ b/tests/qemu-iotests/274.out
@@ -1,9 +1,6 @@
== Commit tests ==
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
wrote 2097152/2097152 bytes at offset 0
2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -66,11 +63,8 @@ read 1048576/1048576 bytes at offset 1048576
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
=== Testing HMP commit (top -> mid) ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
wrote 2097152/2097152 bytes at offset 0
2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -98,11 +92,8 @@ read 1048576/1048576 bytes at offset 1048576
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
=== Testing QMP active commit (top -> mid) ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
wrote 2097152/2097152 bytes at offset 0
2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -136,11 +127,8 @@ read 1048576/1048576 bytes at offset 1048576
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
=== Testing qemu-img commit (top -> base) ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
wrote 2097152/2097152 bytes at offset 0
2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -166,11 +154,8 @@ read 1048576/1048576 bytes at offset 1048576
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
=== Testing QMP active commit (top -> base) ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
wrote 2097152/2097152 bytes at offset 0
2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -205,9 +190,7 @@ read 1048576/1048576 bytes at offset 1048576
== Resize tests ==
=== preallocation=off ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=6442450944 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1073741824 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
wrote 65536/65536 bytes at offset 5368709120
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -224,9 +207,7 @@ read 65536/65536 bytes at offset 5368709120
{ "start": 1073741824, "length": 7516192768, "depth": 0, "present": true, "zero": true, "data": false}]
=== preallocation=metadata ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=34359738368 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=32212254720 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
wrote 65536/65536 bytes at offset 33285996544
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -248,9 +229,7 @@ read 65536/65536 bytes at offset 33285996544
{ "start": 34896609280, "length": 536870912, "depth": 0, "present": true, "zero": true, "data": false, "offset": 2685075456}]
=== preallocation=falloc ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=10485760 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=5242880 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
wrote 65536/65536 bytes at offset 9437184
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -267,9 +246,7 @@ read 65536/65536 bytes at offset 9437184
{ "start": 5242880, "length": 10485760, "depth": 0, "present": true, "zero": false, "data": true, "offset": 327680}]
=== preallocation=full ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=16777216 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=8388608 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
wrote 65536/65536 bytes at offset 11534336
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -286,9 +263,7 @@ read 65536/65536 bytes at offset 11534336
{ "start": 8388608, "length": 4194304, "depth": 0, "present": true, "zero": false, "data": true, "offset": 327680}]
=== preallocation=off ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=393216 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=259072 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
wrote 65536/65536 bytes at offset 259072
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -306,9 +281,7 @@ read 65536/65536 bytes at offset 259072
{ "start": 262144, "length": 262144, "depth": 0, "present": true, "zero": true, "data": false}]
=== preallocation=off ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=409600 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
wrote 65536/65536 bytes at offset 344064
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -325,9 +298,7 @@ read 65536/65536 bytes at offset 344064
{ "start": 262144, "length": 262144, "depth": 0, "present": true, "zero": true, "data": false}]
=== preallocation=off ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=524288 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
wrote 65536/65536 bytes at offset 446464
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
diff --git a/tests/qemu-iotests/280.out b/tests/qemu-iotests/280.out
index 09a0f1a7cb42ff5934e549530e65..e39164c579ff1c1644159e263945 100644
--- a/tests/qemu-iotests/280.out
+++ b/tests/qemu-iotests/280.out
@@ -1,4 +1,3 @@
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
=== Launch VM ===
Enabling migration QMP events on VM...
diff --git a/tests/qemu-iotests/296.out b/tests/qemu-iotests/296.out
index 6c69735604532c161dad12e5d97b..42205cc98141fce2d094755e8224 100644
--- a/tests/qemu-iotests/296.out
+++ b/tests/qemu-iotests/296.out
@@ -1,4 +1,3 @@
-Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
{"execute": "job-dismiss", "arguments": {"id": "job0"}}
{"return": {}}
@@ -13,8 +12,7 @@ Job failed: Failed to get shared "consistent read" lock
qemu-img: Failed to get shared "consistent read" lock
Is another process using the image [TEST_DIR/test.img]?
-.Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
-
+.
Job failed: Block node is read-only
{"execute": "job-dismiss", "arguments": {"id": "job0"}}
{"return": {}}
@@ -26,12 +24,10 @@ Job failed: Failed to get shared "consistent read" lock
{"return": {}}
{"execute": "job-dismiss", "arguments": {"id": "job0"}}
{"return": {}}
-.Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
-
+.
{"return": {}}
{"error": {"class": "GenericError", "desc": "Failed to get \"write\" lock"}}
-.Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
-
+.
{"return": {}}
{"return": {}}
.
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index f424e04573a7f7f07474e5d96f3e..4bb740634df7a0841456aed78c42 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -140,7 +140,9 @@ def unarchive_sample_image(sample, fname):
def qemu_tool_pipe_and_status(tool: str, args: Sequence[str],
- connect_stderr: bool = True) -> Tuple[str, int]:
+ connect_stderr: bool = True,
+ drop_successful_output: bool = False) \
+ -> Tuple[str, int]:
"""
Run a tool and return both its output and its exit code
"""
@@ -152,6 +154,8 @@ def qemu_tool_pipe_and_status(tool: str, args: Sequence[str],
cmd = ' '.join(args)
sys.stderr.write(f'{tool} received signal \
{-subp.returncode}: {cmd}\n')
+ if drop_successful_output and subp.returncode == 0:
+ output = ''
return (output, subp.returncode)
def qemu_img_create_prepare_args(args: List[str]) -> List[str]:
@@ -196,8 +200,10 @@ def qemu_img_pipe_and_status(*args: str) -> Tuple[str, int]:
"""
Run qemu-img and return both its output and its exit code
"""
+ is_create = bool(args and args[0] == 'create')
full_args = qemu_img_args + qemu_img_create_prepare_args(list(args))
- return qemu_tool_pipe_and_status('qemu-img', full_args)
+ return qemu_tool_pipe_and_status('qemu-img', full_args,
+ drop_successful_output=is_create)
def qemu_img(*args: str) -> int:
'''Run qemu-img and return the exit code'''

75
iotests.py-img_info_log-rename-imgopts-a.patch
View File

@ -1,75 +0,0 @@
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Thu, 23 Dec 2021 17:01:26 +0100
Subject: iotests.py: img_info_log(): rename imgopts argument
Git-commit: 3bd2b942d9a8a10bb7a504a1ecf4a3e70803840e
We are going to support IMGOPTS environment variable like in bash
tests. Corresponding global variable in iotests.py should be called
imgopts. So to not interfere with function argument, rename it in
advance.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20211223160144.1097696-2-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
tests/qemu-iotests/210 | 8 ++++----
tests/qemu-iotests/iotests.py | 5 +++--
2 files changed, 7 insertions(+), 6 deletions(-)
diff --git a/tests/qemu-iotests/210 b/tests/qemu-iotests/210
index a4dcc5fe59d8ec9ec66b9cfe4770..10b0a0b87cd1c38c4f675472f560 100755
--- a/tests/qemu-iotests/210
+++ b/tests/qemu-iotests/210
@@ -62,7 +62,7 @@ with iotests.FilePath('t.luks') as disk_path, \
'driver=luks,file.driver=file,file.filename=%s,key-secret=keysec0' % (disk_path),
filter_path=disk_path,
extra_args=['--object', 'secret,id=keysec0,data=foo'],
- imgopts=True)
+ use_image_opts=True)
#
# Successful image creation (with non-default options)
@@ -96,7 +96,7 @@ with iotests.FilePath('t.luks') as disk_path, \
'driver=luks,file.driver=file,file.filename=%s,key-secret=keysec0' % (disk_path),
filter_path=disk_path,
extra_args=['--object', 'secret,id=keysec0,data=foo'],
- imgopts=True)
+ use_image_opts=True)
#
# Invalid BlockdevRef
@@ -132,7 +132,7 @@ with iotests.FilePath('t.luks') as disk_path, \
'driver=luks,file.driver=file,file.filename=%s,key-secret=keysec0' % (disk_path),
filter_path=disk_path,
extra_args=['--object', 'secret,id=keysec0,data=foo'],
- imgopts=True)
+ use_image_opts=True)
#
# Invalid sizes
@@ -176,4 +176,4 @@ with iotests.FilePath('t.luks') as disk_path, \
'driver=luks,file.driver=file,file.filename=%s,key-secret=keysec0' % (disk_path),
filter_path=disk_path,
extra_args=['--object', 'secret,id=keysec0,data=foo'],
- imgopts=True)
+ use_image_opts=True)
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 83bfedb90201ce96118f62dc2b37..9b374dad3dd063fbcbd10963ce5b 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -219,9 +219,10 @@ def qemu_img_log(*args):
log(result, filters=[filter_testfiles])
return result
-def img_info_log(filename, filter_path=None, imgopts=False, extra_args=()):
+def img_info_log(filename, filter_path=None, use_image_opts=False,
+ extra_args=()):
args = ['info']
- if imgopts:
+ if use_image_opts:
args.append('--image-opts')
else:
args += ['-f', imgfmt]

64
iotests.py-implement-unsupported_imgopts.patch
View File

@ -1,64 +0,0 @@
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Thu, 23 Dec 2021 17:01:27 +0100
Subject: iotests.py: implement unsupported_imgopts
Git-commit: 7c15400cdd06b7b9b26c86eac1858fb9c0d77c1c
We are going to support some addition IMGOPTS in python iotests like
in bash iotests. Similarly to bash iotests, we want a way to skip some
tests which can't work with specific IMGOPTS.
Globally for python iotests we will not support things like
'data_file=$TEST_IMG.ext_data_file' in IMGOPTS, so, forbid this
globally in iotests.py.
Suggested-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223160144.1097696-3-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
tests/qemu-iotests/iotests.py | 15 ++++++++++++++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 9b374dad3dd063fbcbd10963ce5b..35d92715ccf4defe721a38982788 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -1215,6 +1215,17 @@ def _verify_virtio_scsi_pci_or_ccw() -> None:
notrun('Missing virtio-scsi-pci or virtio-scsi-ccw in QEMU binary')
+def _verify_imgopts(unsupported: Sequence[str] = ()) -> None:
+ imgopts = os.environ.get('IMGOPTS')
+ # One of usage examples for IMGOPTS is "data_file=$TEST_IMG.ext_data_file"
+ # but it supported only for bash tests. We don't have a concept of global
+ # TEST_IMG in iotests.py, not saying about somehow parsing $variables.
+ # So, for simplicity let's just not support any IMGOPTS with '$' inside.
+ unsup = list(unsupported) + ['$']
+ if imgopts and any(x in imgopts for x in unsup):
+ notrun(f'not suitable for this imgopts: {imgopts}')
+
+
def supports_quorum():
return 'quorum' in qemu_img_pipe('--help')
@@ -1391,7 +1402,8 @@ def execute_setup_common(supported_fmts: Sequence[str] = (),
unsupported_fmts: Sequence[str] = (),
supported_protocols: Sequence[str] = (),
unsupported_protocols: Sequence[str] = (),
- required_fmts: Sequence[str] = ()) -> bool:
+ required_fmts: Sequence[str] = (),
+ unsupported_imgopts: Sequence[str] = ()) -> bool:
"""
Perform necessary setup for either script-style or unittest-style tests.
@@ -1411,6 +1423,7 @@ def execute_setup_common(supported_fmts: Sequence[str] = (),
_verify_aio_mode(supported_aio_modes)
_verify_formats(required_fmts)
_verify_virtio_blk()
+ _verify_imgopts(unsupported_imgopts)
return debug

91
iotests.py-qemu_img-create-support-IMGOP.patch
View File

@ -1,91 +0,0 @@
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Thu, 23 Dec 2021 17:01:29 +0100
Subject: iotests.py: qemu_img*("create"): support
IMGOPTS='compression_type=zstd'
Git-commit: 22e29bcea12ccf0e127b91917d959c69bebbd952
Adding support of IMGOPTS (like in bash tests) allows user to pass a
lot of different options. Still, some may require additional logic.
Now we want compression_type option, so add some smart logic around it:
ignore compression_type=zstd in IMGOPTS, if test want qcow2 in
compatibility mode. As well, ignore compression_type for non-qcow2
formats.
Note that we may instead add support only to qemu_img_create(), but
that works bad:
1. We'll have to update a lot of tests to use qemu_img_create instead
of qemu_img('create'). (still, we may want do it anyway, but no
reason to create a dependancy between task of supporting IMGOPTS and
updating a lot of tests)
2. Some tests use qemu_img_pipe('create', ..) - even more work on
updating
3. Even if we update all tests to go through qemu_img_create, we'll
need a way to avoid creating new tests using qemu_img*('create') -
add assertions.. That doesn't seem good.
So, let's add support of IMGOPTS to most generic
qemu_img_pipe_and_status().
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223160144.1097696-5-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
tests/qemu-iotests/iotests.py | 27 ++++++++++++++++++++++++++-
1 file changed, 26 insertions(+), 1 deletion(-)
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 35d92715ccf4defe721a38982788..f0370b520651aa4ccc1cdb0cf28e 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -16,6 +16,7 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
+import argparse
import atexit
import bz2
from collections import OrderedDict
@@ -153,11 +154,35 @@ def qemu_tool_pipe_and_status(tool: str, args: Sequence[str],
{-subp.returncode}: {cmd}\n')
return (output, subp.returncode)
+def qemu_img_create_prepare_args(args: List[str]) -> List[str]:
+ if not args or args[0] != 'create':
+ return list(args)
+ args = args[1:]
+
+ p = argparse.ArgumentParser(allow_abbrev=False)
+ p.add_argument('-f')
+ parsed, remaining = p.parse_known_args(args)
+
+ result = ['create']
+ if parsed.f is not None:
+ result += ['-f', parsed.f]
+
+ # IMGOPTS most probably contain options specific for the selected format,
+ # like extended_l2 or compression_type for qcow2. Test may want to create
+ # additional images in other formats that doesn't support these options.
+ # So, use IMGOPTS only for images created in imgfmt format.
+ if parsed.f == imgfmt and 'IMGOPTS' in os.environ:
+ result += ['-o', os.environ['IMGOPTS']]
+
+ result += remaining
+
+ return result
+
def qemu_img_pipe_and_status(*args: str) -> Tuple[str, int]:
"""
Run qemu-img and return both its output and its exit code
"""
- full_args = qemu_img_args + list(args)
+ full_args = qemu_img_args + qemu_img_create_prepare_args(list(args))
return qemu_tool_pipe_and_status('qemu-img', full_args)
def qemu_img(*args: str) -> int:

83
iotests.py-rewrite-default-luks-support-.patch
View File

@ -1,83 +0,0 @@
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Thu, 23 Dec 2021 17:01:31 +0100
Subject: iotests.py: rewrite default luks support in qemu_img
Git-commit: 28a5ad93da08ae55c8dfac0db8615936ca14b822
Move the logic to more generic qemu_img_pipe_and_status(). Also behave
better when we have several -o options. And reuse argument parser of
course.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223160144.1097696-7-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
tests/qemu-iotests/iotests.py | 36 +++++++++++++++++------------------
1 file changed, 17 insertions(+), 19 deletions(-)
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 5a6bfcbb6adbb84d412e37966360..f424e04573a7f7f07474e5d96f3e 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -160,9 +160,13 @@ def qemu_img_create_prepare_args(args: List[str]) -> List[str]:
args = args[1:]
p = argparse.ArgumentParser(allow_abbrev=False)
+ # -o option may be specified several times
+ p.add_argument('-o', action='append', default=[])
p.add_argument('-f')
parsed, remaining = p.parse_known_args(args)
+ opts_list = parsed.o
+
result = ['create']
if parsed.f is not None:
result += ['-f', parsed.f]
@@ -171,8 +175,18 @@ def qemu_img_create_prepare_args(args: List[str]) -> List[str]:
# like extended_l2 or compression_type for qcow2. Test may want to create
# additional images in other formats that doesn't support these options.
# So, use IMGOPTS only for images created in imgfmt format.
- if parsed.f == imgfmt and 'IMGOPTS' in os.environ:
- result += ['-o', os.environ['IMGOPTS']]
+ imgopts = os.environ.get('IMGOPTS')
+ if imgopts and parsed.f == imgfmt:
+ opts_list.insert(0, imgopts)
+
+ # default luks support
+ if parsed.f == 'luks' and \
+ all('key-secret' not in opts for opts in opts_list):
+ result += ['--object', luks_default_secret_object]
+ opts_list.append(luks_default_key_secret_opt)
+
+ for opts in opts_list:
+ result += ['-o', opts]
result += remaining
@@ -203,23 +217,7 @@ def ordered_qmp(qmsg, conv_keys=True):
return qmsg
def qemu_img_create(*args):
- args = list(args)
-
- # default luks support
- if '-f' in args and args[args.index('-f') + 1] == 'luks':
- if '-o' in args:
- i = args.index('-o')
- if 'key-secret' not in args[i + 1]:
- args[i + 1].append(luks_default_key_secret_opt)
- args.insert(i + 2, '--object')
- args.insert(i + 3, luks_default_secret_object)
- else:
- args = ['-o', luks_default_key_secret_opt,
- '--object', luks_default_secret_object] + args
-
- args.insert(0, 'create')
-
- return qemu_img(*args)
+ return qemu_img('create', *args)
def qemu_img_measure(*args):
return json.loads(qemu_img_pipe("measure", "--output", "json", *args))

6
linux-user-Fake-proc-cpuinfo.patch
View File

@ -21,10 +21,10 @@ Signed-off-by: Andreas Färber <afaerber@suse.de>
1 file changed, 24 insertions(+)
diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 0a1d99cb44d168d561de4c42e132..d4f3295b9d1837126f35d8357e80 100644
index c9adb65a2293e9735341a217a68d..8de1bb8d48eeb867a1047d986299 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -7871,6 +7871,27 @@ static int open_self_stat(void *cpu_env, int fd)
@@ -8099,6 +8099,27 @@ static int open_self_stat(void *cpu_env, int fd)
return 0;
}
@ -52,7 +52,7 @@ index 0a1d99cb44d168d561de4c42e132..d4f3295b9d1837126f35d8357e80 100644
static int open_self_auxv(void *cpu_env, int fd)
{
CPUState *cpu = env_cpu((CPUArchState *)cpu_env);
@@ -8025,6 +8046,9 @@ static int do_openat(void *cpu_env, int dirfd, const char *pathname, int flags,
@@ -8253,6 +8274,9 @@ static int do_openat(void *cpu_env, int dirfd, const char *pathname, int flags,
#if defined(TARGET_SPARC) || defined(TARGET_HPPA)
{ "/proc/cpuinfo", open_cpuinfo, is_proc },
#endif

4
linux-user-add-binfmt-wrapper-for-argv-0.patch
View File

@ -82,10 +82,10 @@ index 0000000000000000000000000000000000000000..cd1f513b334f3b263d9e4b5adb1981e3
+ return execve(new_argv[0], new_argv, envp);
+}
diff --git a/meson.build b/meson.build
index 96de1a6ef948542aa93bd0324200..612da7167f5c4aafcf77a0e1ce55 100644
index 861de93c4f58f294f9d353c9674e..05740dd14228fb700ecf6b793cfc 100644
--- a/meson.build
+++ b/meson.build
@@ -3043,6 +3043,11 @@ endforeach
@@ -3336,6 +3336,11 @@ endforeach
# Other build targets

4
linux-user-lseek-explicitly-cast-non-set.patch
View File

@ -15,10 +15,10 @@ Signed-off-by: Alexander Graf <agraf@suse.de>
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 381066e788eb36c1d6ca5b872353..7fe3a69c7cd860d7c1a8ea88a48d 100644
index 5096b1258f6d631b80fb6cca6011..ae26ad25466aca617d272e4c7474 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -8701,8 +8701,13 @@ static abi_long do_syscall1(void *cpu_env, int num, abi_ulong arg1,
@@ -8929,8 +8929,13 @@ static abi_long do_syscall1(void *cpu_env, int num, abi_ulong arg1,
return ret;
#endif
#ifdef TARGET_NR_lseek

38
linux-user-use-target_ulong.patch
View File

@ -12,16 +12,29 @@ Pass syscall arguments as ulong always.
Signed-off-by: Alexander Graf <agraf@suse.de>
[JRZ: changes from linux-user/qemu.h wass moved to linux-user/user-internals.h]
Signed-off-by: Jose R Ziviani <jziviani@suse.de>
[DF: Forward port, i.e., use ulong for do_prctl too]
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
linux-user/syscall.c | 18 +++++++++---------
linux-user/syscall.c | 20 ++++++++++----------
linux-user/user-internals.h | 8 ++++----
2 files changed, 13 insertions(+), 13 deletions(-)
2 files changed, 14 insertions(+), 14 deletions(-)
diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index d4f3295b9d1837126f35d8357e80..381066e788eb36c1d6ca5b872353 100644
index 8de1bb8d48eeb867a1047d986299..5096b1258f6d631b80fb6cca6011 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -8346,10 +8346,10 @@ _syscall2(int, pivot_root, const char *, new_root, const char *, put_old)
@@ -6406,8 +6406,8 @@ static abi_long do_prctl_inval1(CPUArchState *env, abi_long arg2)
#define do_prctl_set_unalign do_prctl_inval1
#endif
-static abi_long do_prctl(CPUArchState *env, abi_long option, abi_long arg2,
- abi_long arg3, abi_long arg4, abi_long arg5)
+static abi_long do_prctl(CPUArchState *env, abi_ulong option, abi_ulong arg2,
+ abi_ulong arg3, abi_ulong arg4, abi_ulong arg5)
{
abi_long ret;
@@ -8574,10 +8574,10 @@ _syscall2(int, pivot_root, const char *, new_root, const char *, put_old)
* of syscall results, can be performed.
* All errnos that do_syscall() returns must be -TARGET_<errcode>.
*/
@ -36,16 +49,7 @@ index d4f3295b9d1837126f35d8357e80..381066e788eb36c1d6ca5b872353 100644
{
CPUState *cpu = env_cpu(cpu_env);
abi_long ret;
@@ -10807,7 +10807,7 @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1,
*/
ret = -TARGET_EINVAL;
if (cpu_isar_feature(aa64_sve, env_archcpu(cpu_env))
- && arg2 >= 0 && arg2 <= 512 * 16 && !(arg2 & 15)) {
+ && arg2 <= 512 * 16 && !(arg2 & 15)) {
CPUARMState *env = cpu_env;
ARMCPU *cpu = env_archcpu(env);
uint32_t vq, old_vq;
@@ -13176,10 +13176,10 @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1,
@@ -13201,10 +13201,10 @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1,
return ret;
}
@ -61,10 +65,10 @@ index d4f3295b9d1837126f35d8357e80..381066e788eb36c1d6ca5b872353 100644
CPUState *cpu = env_cpu(cpu_env);
abi_long ret;
diff --git a/linux-user/user-internals.h b/linux-user/user-internals.h
index 661612a088b5c4e37f8f9fbcb6af..db24553432003b2faa3957d63c3d 100644
index ee152ccfaa8fdfa33b1e6d36b8ab..7117d8b2c583a16311ef82a0eb60 100644
--- a/linux-user/user-internals.h
+++ b/linux-user/user-internals.h
@@ -60,10 +60,10 @@ int info_is_fdpic(struct image_info *info);
@@ -59,10 +59,10 @@ int info_is_fdpic(struct image_info *info);
void target_set_brk(abi_ulong new_brk);
void syscall_init(void);
@ -77,5 +81,5 @@ index 661612a088b5c4e37f8f9fbcb6af..db24553432003b2faa3957d63c3d 100644
+ abi_ulong arg5, abi_ulong arg6, abi_ulong arg7,
+ abi_ulong arg8);
extern __thread CPUState *thread_cpu;
void cpu_loop(CPUArchState *env);
void QEMU_NORETURN cpu_loop(CPUArchState *env);
const char *target_strerror(int err);

54
meson-build-all-modules-by-default.patch
View File

@ -1,54 +0,0 @@
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 7 Jan 2022 13:00:44 +0100
Subject: meson: build all modules by default
Git-commit: fb72176ba64057ce0ee5c1a0a443d9f72e270636
With more recent versions of Meson, the build.ninja file is more selective
as to what is built by default, and not building the modules results in test
failures.
Mark the modules as built-by-default and, to make the dependencies more
precise, also require them to be up-to-date before running tests.
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/801
Tested-by: Li Zhang <lizhang@suse.de>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
meson.build | 4 +++-
tests/qtest/meson.build | 3 +--
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/meson.build b/meson.build
index 612da7167f5c4aafcf77a0e1ce55..4997739df656a10fa1822116102a 100644
--- a/meson.build
+++ b/meson.build
@@ -2818,8 +2818,10 @@ common_ss.add(hwcore)
# Targets #
###########
+emulator_modules = []
foreach m : block_mods + softmmu_mods
- shared_module(m.name(),
+ emulator_modules += shared_module(m.name(),
+ build_by_default: true,
name_prefix: '',
link_whole: m,
install: true,
diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
index c9d8458062ff6b88d1c28cb21e8f..6aff75bafa5761da141f27e6b1ee 100644
--- a/tests/qtest/meson.build
+++ b/tests/qtest/meson.build
@@ -299,10 +299,9 @@ foreach dir : target_dirs
test: executable(test, src, dependencies: deps)
}
endif
- # FIXME: missing dependency on the emulator binary and qemu-img
test('qtest-@0@/@1@'.format(target_base, test),
qtest_executables[test],
- depends: [test_deps, qtest_emulator],
+ depends: [test_deps, qtest_emulator, emulator_modules],
env: qtest_env,
args: ['--tap', '-k'],
protocol: 'tap',

8
meson-install-ivshmem-client-and-ivshmem.patch
View File

@ -11,21 +11,21 @@ Signed-off-by: Bruce Rogers <brogers@suse.com>
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/contrib/ivshmem-client/meson.build b/contrib/ivshmem-client/meson.build
index 1b171efb4f536f608191cd4cba1b..83a559117fd1a2c99f34a43fa38d 100644
index ce8dcca84ddb23de1d8eaa813f5a..bc77336258302e0f7a962ef47ed6 100644
--- a/contrib/ivshmem-client/meson.build
+++ b/contrib/ivshmem-client/meson.build
@@ -1,4 +1,4 @@
executable('ivshmem-client', files('ivshmem-client.c', 'main.c'),
executable('ivshmem-client', files('ivshmem-client.c', 'main.c'), genh,
dependencies: glib,
build_by_default: targetos == 'linux',
- install: false)
+ install: true)
diff --git a/contrib/ivshmem-server/meson.build b/contrib/ivshmem-server/meson.build
index 3a539422017e6dc342e098c3e9c8..a1c39aa3b30e9c56941cbc394f86 100644
index c6c3c82e89f363e1929bfa08e318..185879a0c6ce71b1272a7ef6ce4d 100644
--- a/contrib/ivshmem-server/meson.build
+++ b/contrib/ivshmem-server/meson.build
@@ -1,4 +1,4 @@
executable('ivshmem-server', files('ivshmem-server.c', 'main.c'),
executable('ivshmem-server', files('ivshmem-server.c', 'main.c'), genh,
dependencies: [qemuutil, rt],
build_by_default: targetos == 'linux',
- install: false)

137
modules-generates-per-target-modinfo.patch Normal file
View File

@ -0,0 +1,137 @@
From: "Jose R. Ziviani" <jziviani@suse.de>
Date: Tue, 28 Sep 2021 17:46:28 -0300
Subject: modules: generates per-target modinfo
Git-commit: 05d6814c3eb16524e992bb7048d3385f8e99dd6a
References: bsc#1199015
This patch changes the way modinfo is generated and built. Instead of
one modinfo.c it generates one modinfo-<target>-softmmu.c per target. It
aims a fine-tune control of modules by configuring Kconfig.
Signed-off-by: Jose R. Ziviani <jziviani@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Message-Id: <165369003038.5857.13084289285185196779.stgit@work>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
meson.build | 25 +++++++++++++++-------
scripts/modinfo-generate.py | 42 ++++++++++++++++++++++---------------
2 files changed, 42 insertions(+), 25 deletions(-)
diff --git a/meson.build b/meson.build
index 05740dd14228fb700ecf6b793cfc..eeaa045c90402cee5ba946f321c1 100644
--- a/meson.build
+++ b/meson.build
@@ -2999,14 +2999,23 @@ foreach d, list : target_modules
endforeach
if enable_modules
- modinfo_src = custom_target('modinfo.c',
- output: 'modinfo.c',
- input: modinfo_files,
- command: [modinfo_generate, '@INPUT@'],
- capture: true)
- modinfo_lib = static_library('modinfo', modinfo_src)
- modinfo_dep = declare_dependency(link_whole: modinfo_lib)
- softmmu_ss.add(modinfo_dep)
+ foreach target : target_dirs
+ if target.endswith('-softmmu')
+ config_target = config_target_mak[target]
+ config_devices_mak = target + '-config-devices.mak'
+ modinfo_src = custom_target('modinfo-' + target + '.c',
+ output: 'modinfo-' + target + '.c',
+ input: modinfo_files,
+ command: [modinfo_generate, '--devices', config_devices_mak, '@INPUT@'],
+ capture: true)
+
+ modinfo_lib = static_library('modinfo-' + target + '.c', modinfo_src)
+ modinfo_dep = declare_dependency(link_with: modinfo_lib)
+
+ arch = config_target['TARGET_NAME'] == 'sparc64' ? 'sparc64' : config_target['TARGET_BASE_ARCH']
+ hw_arch[arch].add(modinfo_dep)
+ endif
+ endforeach
endif
nm = find_program('nm')
diff --git a/scripts/modinfo-generate.py b/scripts/modinfo-generate.py
index 689f33c0f2919508341712a1cde4..a0c09edae1009bc77964e95d585f 100755
--- a/scripts/modinfo-generate.py
+++ b/scripts/modinfo-generate.py
@@ -32,7 +32,7 @@ def parse_line(line):
continue
return (kind, data)
-def generate(name, lines):
+def generate(name, lines, core_modules):
arch = ""
objs = []
deps = []
@@ -49,7 +49,13 @@ def generate(name, lines):
elif kind == 'arch':
arch = data;
elif kind == 'kconfig':
- pass # ignore
+ # don't add a module which dependency is not enabled
+ # in kconfig
+ if data.strip() not in core_modules:
+ print(" /* module {} isn't enabled in Kconfig. */"
+ .format(data.strip()))
+ print("/* },{ */")
+ return []
else:
print("unknown:", kind)
exit(1)
@@ -60,7 +66,7 @@ def generate(name, lines):
print_array("objs", objs)
print_array("deps", deps)
print_array("opts", opts)
- print("},{");
+ print("},{")
return deps
def print_pre():
@@ -74,26 +80,28 @@ def print_post():
print("}};")
def main(args):
+ if len(args) < 3 or args[0] != '--devices':
+ print('Expected: modinfo-generate.py --devices '
+ 'config-device.mak [modinfo files]', file=sys.stderr)
+ exit(1)
+
+ # get all devices enabled in kconfig, from *-config-device.mak
+ enabled_core_modules = set()
+ with open(args[1]) as file:
+ for line in file.readlines():
+ config = line.split('=')
+ if config[1].rstrip() == 'y':
+ enabled_core_modules.add(config[0][7:]) # remove CONFIG_
+
deps = {}
print_pre()
- for modinfo in args:
+ for modinfo in args[2:]:
with open(modinfo) as f:
lines = f.readlines()
print(" /* %s */" % modinfo)
- (basename, ext) = os.path.splitext(modinfo)
- deps[basename] = generate(basename, lines)
+ (basename, _) = os.path.splitext(modinfo)
+ deps[basename] = generate(basename, lines, enabled_core_modules)
print_post()
- flattened_deps = {flat.strip('" ') for dep in deps.values() for flat in dep}
- error = False
- for dep in flattened_deps:
- if dep not in deps.keys():
- print("Dependency {} cannot be satisfied".format(dep),
- file=sys.stderr)
- error = True
-
- if error:
- exit(1)
-
if __name__ == "__main__":
main(sys.argv[1:])

269
modules-introduces-module_kconfig-direct.patch Normal file
View File

@ -0,0 +1,269 @@
From: "Jose R. Ziviani" <jziviani@suse.de>
Date: Tue, 28 Sep 2021 17:46:27 -0300
Subject: modules: introduces module_kconfig directive
Git-commit: 24ce7aa77dd1a3095d994bf53d60cce6e672cf4d
References: bsc#1199015
module_kconfig is a new directive that should be used with module_obj
whenever that module depends on the Kconfig to be enabled.
When the module is enabled in Kconfig we are sure that its dependencies
will be enabled as well, thus the module will be loaded without any
problem.
The correct way to use module_kconfig is by passing the Kconfig option
to module_kconfig (or the *config-devices.mak without CONFIG_).
Signed-off-by: Jose R. Ziviani <jziviani@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Message-Id: <165369002370.5857.12150544416563557322.stgit@work>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
hw/display/qxl.c | 1 +
hw/display/vhost-user-gpu-pci.c | 1 +
hw/display/vhost-user-gpu.c | 1 +
hw/display/vhost-user-vga.c | 1 +
hw/display/virtio-gpu-base.c | 1 +
hw/display/virtio-gpu-gl.c | 1 +
hw/display/virtio-gpu-pci-gl.c | 1 +
hw/display/virtio-gpu-pci.c | 1 +
hw/display/virtio-gpu.c | 1 +
hw/display/virtio-vga-gl.c | 1 +
hw/display/virtio-vga.c | 1 +
hw/s390x/virtio-ccw-gpu.c | 1 +
hw/usb/ccid-card-emulated.c | 1 +
hw/usb/ccid-card-passthru.c | 1 +
hw/usb/host-libusb.c | 1 +
hw/usb/redirect.c | 1 +
include/qemu/module.h | 10 ++++++++++
scripts/modinfo-generate.py | 2 ++
18 files changed, 28 insertions(+)
diff --git a/hw/display/qxl.c b/hw/display/qxl.c
index adbdbcaeb6121106bfcaee288d4c..092fb36b3a928b062067207d7518 100644
--- a/hw/display/qxl.c
+++ b/hw/display/qxl.c
@@ -2515,6 +2515,7 @@ static const TypeInfo qxl_primary_info = {
.class_init = qxl_primary_class_init,
};
module_obj("qxl-vga");
+module_kconfig(QXL);
static void qxl_secondary_class_init(ObjectClass *klass, void *data)
{
diff --git a/hw/display/vhost-user-gpu-pci.c b/hw/display/vhost-user-gpu-pci.c
index daefcf7101590cbd3b3681ac6fa9..d119bcae45d48e6eb14afe7d26e6 100644
--- a/hw/display/vhost-user-gpu-pci.c
+++ b/hw/display/vhost-user-gpu-pci.c
@@ -44,6 +44,7 @@ static const VirtioPCIDeviceTypeInfo vhost_user_gpu_pci_info = {
.instance_init = vhost_user_gpu_pci_initfn,
};
module_obj(TYPE_VHOST_USER_GPU_PCI);
+module_kconfig(VHOST_USER_GPU);
static void vhost_user_gpu_pci_register_types(void)
{
diff --git a/hw/display/vhost-user-gpu.c b/hw/display/vhost-user-gpu.c
index 09818231bd24a9f8acd7ee3c2e15..5fc192022346dfc7ad5646db069d 100644
--- a/hw/display/vhost-user-gpu.c
+++ b/hw/display/vhost-user-gpu.c
@@ -599,6 +599,7 @@ static const TypeInfo vhost_user_gpu_info = {
.class_init = vhost_user_gpu_class_init,
};
module_obj(TYPE_VHOST_USER_GPU);
+module_kconfig(VHOST_USER_GPU);
static void vhost_user_gpu_register_types(void)
{
diff --git a/hw/display/vhost-user-vga.c b/hw/display/vhost-user-vga.c
index 072c9c65bc754f399bc49a3dbb88..0c146080fd2c15ff9fd86338164e 100644
--- a/hw/display/vhost-user-vga.c
+++ b/hw/display/vhost-user-vga.c
@@ -45,6 +45,7 @@ static const VirtioPCIDeviceTypeInfo vhost_user_vga_info = {
.instance_init = vhost_user_vga_inst_initfn,
};
module_obj(TYPE_VHOST_USER_VGA);
+module_kconfig(VHOST_USER_VGA);
static void vhost_user_vga_register_types(void)
{
diff --git a/hw/display/virtio-gpu-base.c b/hw/display/virtio-gpu-base.c
index fff0fb4a828a0814ab5290a17f78..a8a5a8812f331fa56e3ef6a18821 100644
--- a/hw/display/virtio-gpu-base.c
+++ b/hw/display/virtio-gpu-base.c
@@ -260,6 +260,7 @@ static const TypeInfo virtio_gpu_base_info = {
.abstract = true
};
module_obj(TYPE_VIRTIO_GPU_BASE);
+module_kconfig(VIRTIO_GPU);
static void
virtio_register_types(void)
diff --git a/hw/display/virtio-gpu-gl.c b/hw/display/virtio-gpu-gl.c
index 6cc4313b1af2c3fac0011d6b39aa..f7837cc44dc41c665b703d12c79f 100644
--- a/hw/display/virtio-gpu-gl.c
+++ b/hw/display/virtio-gpu-gl.c
@@ -160,6 +160,7 @@ static const TypeInfo virtio_gpu_gl_info = {
.class_init = virtio_gpu_gl_class_init,
};
module_obj(TYPE_VIRTIO_GPU_GL);
+module_kconfig(VIRTIO_GPU);
static void virtio_register_types(void)
{
diff --git a/hw/display/virtio-gpu-pci-gl.c b/hw/display/virtio-gpu-pci-gl.c
index 99b14a07185ea6b3a2a2f99f167f..a2819e1ca93f98022a81b5a14d18 100644
--- a/hw/display/virtio-gpu-pci-gl.c
+++ b/hw/display/virtio-gpu-pci-gl.c
@@ -47,6 +47,7 @@ static const VirtioPCIDeviceTypeInfo virtio_gpu_gl_pci_info = {
.instance_init = virtio_gpu_gl_initfn,
};
module_obj(TYPE_VIRTIO_GPU_GL_PCI);
+module_kconfig(VIRTIO_PCI);
static void virtio_gpu_gl_pci_register_types(void)
{
diff --git a/hw/display/virtio-gpu-pci.c b/hw/display/virtio-gpu-pci.c
index e36eee0c409bb8d3bf7cbec85278..93f214ff58127452d0d56a19c361 100644
--- a/hw/display/virtio-gpu-pci.c
+++ b/hw/display/virtio-gpu-pci.c
@@ -65,6 +65,7 @@ static const TypeInfo virtio_gpu_pci_base_info = {
.abstract = true
};
module_obj(TYPE_VIRTIO_GPU_PCI_BASE);
+module_kconfig(VIRTIO_PCI);
#define TYPE_VIRTIO_GPU_PCI "virtio-gpu-pci"
typedef struct VirtIOGPUPCI VirtIOGPUPCI;
diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
index 529b5246b2b55da16445712e4173..cd4a56056fd9407a904470878318 100644
--- a/hw/display/virtio-gpu.c
+++ b/hw/display/virtio-gpu.c
@@ -1452,6 +1452,7 @@ static const TypeInfo virtio_gpu_info = {
.class_init = virtio_gpu_class_init,
};
module_obj(TYPE_VIRTIO_GPU);
+module_kconfig(VIRTIO_GPU);
static void virtio_register_types(void)
{
diff --git a/hw/display/virtio-vga-gl.c b/hw/display/virtio-vga-gl.c
index f22549097c5e8bb070da6d1c28d4..984faa6b39a80bc786e31ec3d393 100644
--- a/hw/display/virtio-vga-gl.c
+++ b/hw/display/virtio-vga-gl.c
@@ -37,6 +37,7 @@ static VirtioPCIDeviceTypeInfo virtio_vga_gl_info = {
.instance_init = virtio_vga_gl_inst_initfn,
};
module_obj(TYPE_VIRTIO_VGA_GL);
+module_kconfig(VIRTIO_VGA);
static void virtio_vga_register_types(void)
{
diff --git a/hw/display/virtio-vga.c b/hw/display/virtio-vga.c
index 7b55c8d0e72390b430fc4ad30f9f..c206b5da384bcc18dd14d3fcf62a 100644
--- a/hw/display/virtio-vga.c
+++ b/hw/display/virtio-vga.c
@@ -231,6 +231,7 @@ static const TypeInfo virtio_vga_base_info = {
.abstract = true,
};
module_obj(TYPE_VIRTIO_VGA_BASE);
+module_kconfig(VIRTIO_VGA);
#define TYPE_VIRTIO_VGA "virtio-vga"
diff --git a/hw/s390x/virtio-ccw-gpu.c b/hw/s390x/virtio-ccw-gpu.c
index 5868a2a07093f163f93690b9b87b..024e8990f499b0b95f8912da86d5 100644
--- a/hw/s390x/virtio-ccw-gpu.c
+++ b/hw/s390x/virtio-ccw-gpu.c
@@ -60,6 +60,7 @@ static const TypeInfo virtio_ccw_gpu = {
.class_init = virtio_ccw_gpu_class_init,
};
module_obj(TYPE_VIRTIO_GPU_CCW);
+module_kconfig(VIRTIO_CCW);
static void virtio_ccw_gpu_register(void)
{
diff --git a/hw/usb/ccid-card-emulated.c b/hw/usb/ccid-card-emulated.c
index 6c8c0355e099c94a84fdf3b4ab12..1ddf7297f6125d59eae04a2ef5c4 100644
--- a/hw/usb/ccid-card-emulated.c
+++ b/hw/usb/ccid-card-emulated.c
@@ -613,6 +613,7 @@ static const TypeInfo emulated_card_info = {
.class_init = emulated_class_initfn,
};
module_obj(TYPE_EMULATED_CCID);
+module_kconfig(USB);
static void ccid_card_emulated_register_types(void)
{
diff --git a/hw/usb/ccid-card-passthru.c b/hw/usb/ccid-card-passthru.c
index fa3040fb7154419ce9767d55e4ed..748544d170835ca5a1ef99c1faac 100644
--- a/hw/usb/ccid-card-passthru.c
+++ b/hw/usb/ccid-card-passthru.c
@@ -415,6 +415,7 @@ static const TypeInfo passthru_card_info = {
.class_init = passthru_class_initfn,
};
module_obj(TYPE_CCID_PASSTHRU);
+module_kconfig(USB);
static void ccid_card_passthru_register_types(void)
{
diff --git a/hw/usb/host-libusb.c b/hw/usb/host-libusb.c
index 2b35cb6cdd3933f2cf98307bcc4e..28f8af8941ff6db20283fcd85d23 100644
--- a/hw/usb/host-libusb.c
+++ b/hw/usb/host-libusb.c
@@ -1809,6 +1809,7 @@ static const TypeInfo usb_host_dev_info = {
.instance_init = usb_host_instance_init,
};
module_obj(TYPE_USB_HOST_DEVICE);
+module_kconfig(USB);
static void usb_host_register_types(void)
{
diff --git a/hw/usb/redirect.c b/hw/usb/redirect.c
index 8692ea256109532acf791bb590bd..b384ad7e73fbea58655324a65fb8 100644
--- a/hw/usb/redirect.c
+++ b/hw/usb/redirect.c
@@ -2620,6 +2620,7 @@ static const TypeInfo usbredir_dev_info = {
.instance_init = usbredir_instance_init,
};
module_obj(TYPE_USB_REDIR);
+module_kconfig(USB);
static void usbredir_register_types(void)
{
diff --git a/include/qemu/module.h b/include/qemu/module.h
index 5fcc323b2a79d5adfdf27fa19bf7..bd73607104c951a34512a156cff0 100644
--- a/include/qemu/module.h
+++ b/include/qemu/module.h
@@ -135,6 +135,16 @@ void module_allow_arch(const char *arch);
*/
#define module_opts(name) modinfo(opts, name)
+/**
+ * module_kconfig
+ *
+ * @name: Kconfig requirement necessary to load the module
+ *
+ * This module requires a core module that should be implemented and
+ * enabled in Kconfig.
+ */
+#define module_kconfig(name) modinfo(kconfig, name)
+
/*
* module info database
*
diff --git a/scripts/modinfo-generate.py b/scripts/modinfo-generate.py
index f559eed0077a22bfd25a26a79c01..689f33c0f2919508341712a1cde4 100755
--- a/scripts/modinfo-generate.py
+++ b/scripts/modinfo-generate.py
@@ -48,6 +48,8 @@ def generate(name, lines):
opts.append(data)
elif kind == 'arch':
arch = data;
+ elif kind == 'kconfig':
+ pass # ignore
else:
print("unknown:", kind)
exit(1)

88
modules-quick-fix-a-fundamental-error-in.patch
View File

@ -1,88 +0,0 @@
From: "Jose R. Ziviani" <jziviani@suse.de>
Date: Thu, 16 Sep 2021 00:52:34 -0300
Subject: modules: quick-fix a fundamental error in modules
Git-commit: 00000000000000000000000000000000000000000
References: bsc#1190573
modinfo.c is generated regarding the arch being built. However, if we
build multiple arch targets at once, modinfo.c will list modules that
might not be supported by all these targets. When trying to run these
targets, errors[1] will appear.
This patch is a bandaid specific to s390x. It will be removed as soon
as a better approach is fixed upstream.
[1]
$ qemu-system-s390x -nodefaults -display none -accel qtest -M none -device help
Failed to open module: /home/jose/qemu/build/hw-display-virtio-vga.so: undefined symbol: vmstate_vga_common
Signed-off-by: Jose R. Ziviani <jziviani@suse.de>
---
include/qemu/module.h | 1 +
util/module.c | 33 +++++++++++++++++++++++++++++++++
2 files changed, 34 insertions(+)
diff --git a/include/qemu/module.h b/include/qemu/module.h
index 5fcc323b2a79d5adfdf27fa19bf7..ed051a6c0e7df56015e25936e641 100644
--- a/include/qemu/module.h
+++ b/include/qemu/module.h
@@ -73,6 +73,7 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail);
void module_load_qom_one(const char *type);
void module_load_qom_all(void);
void module_allow_arch(const char *arch);
+bool s390x_blocklist(const char *name);
/**
* DOC: module info annotation macros
diff --git a/util/module.c b/util/module.c
index 6bb4ad915a1c86f1601a4913a29c..8370d4c6209eda4319342b202f6b 100644
--- a/util/module.c
+++ b/util/module.c
@@ -119,6 +119,35 @@ static const QemuModinfo module_info_stub[] = { {
static const QemuModinfo *module_info = module_info_stub;
static const char *module_arch;
+bool s390x_blocklist(const char *name)
+{
+ const char *blocklist[] = {
+ "hw-display-qxl",
+ "hw-display-virtio-vga",
+ "hw-display-virtio-vga-gl",
+ "hw-usb-host",
+ "hw-usb-redirect",
+ "hw-usb-smartcard"
+ };
+
+ const size_t len = sizeof(blocklist) / sizeof(blocklist[0]);
+
+ if (strcmp(module_arch, "x86_64") == 0 ||
+ strcmp(module_arch, "i386") == 0 ||
+ strcmp(module_arch, "arm") == 0 ||
+ strcmp(module_arch, "aarch64") == 0) {
+ return false;
+ }
+
+ for (size_t i = 0; i < len; i++) {
+ if (strcmp(blocklist[i], name) == 0) {
+ return true;
+ }
+ }
+
+ return false;
+}
+
void module_init_info(const QemuModinfo *info)
{
module_info = info;
@@ -131,6 +160,10 @@ void module_allow_arch(const char *arch)
static bool module_check_arch(const QemuModinfo *modinfo)
{
+ if (modinfo->name && s390x_blocklist(modinfo->name)) {
+ return false;
+ }
+
if (modinfo->arch) {
if (!module_arch) {
/* no arch set -> ignore all */

277
numa-Enable-numa-for-SGX-EPC-sections.patch
View File

@ -1,277 +0,0 @@
From: Yang Zhong <yang.zhong@intel.com>
Date: Mon, 1 Nov 2021 12:20:05 -0400
Subject: numa: Enable numa for SGX EPC sections
Git-commit: 1105812382e1126d86dddc16b3700f8c79dc93d1
References: bsc#1197807
The basic SGX did not enable numa for SGX EPC sections, which
result in all EPC sections located in numa node 0. This patch
enable SGX numa function in the guest and the EPC section can
work with RAM as one numa node.
The Guest kernel related log:
[ 0.009981] ACPI: SRAT: Node 0 PXM 0 [mem 0x180000000-0x183ffffff]
[ 0.009982] ACPI: SRAT: Node 1 PXM 1 [mem 0x184000000-0x185bfffff]
The SRAT table can normally show SGX EPC sections menory info in different
numa nodes.
The SGX EPC numa related command:
......
-m 4G,maxmem=20G \
-smp sockets=2,cores=2 \
-cpu host,+sgx-provisionkey \
-object memory-backend-ram,size=2G,host-nodes=0,policy=bind,id=node0 \
-object memory-backend-epc,id=mem0,size=64M,prealloc=on,host-nodes=0,policy=bind \
-numa node,nodeid=0,cpus=0-1,memdev=node0 \
-object memory-backend-ram,size=2G,host-nodes=1,policy=bind,id=node1 \
-object memory-backend-epc,id=mem1,size=28M,prealloc=on,host-nodes=1,policy=bind \
-numa node,nodeid=1,cpus=2-3,memdev=node1 \
-M sgx-epc.0.memdev=mem0,sgx-epc.0.node=0,sgx-epc.1.memdev=mem1,sgx-epc.1.node=1 \
......
Signed-off-by: Yang Zhong <yang.zhong@intel.com>
Message-Id: <20211101162009.62161-2-yang.zhong@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
hw/core/numa.c | 5 ++---
hw/i386/acpi-build.c | 2 ++
hw/i386/sgx-epc.c | 3 +++
hw/i386/sgx-stub.c | 4 ++++
hw/i386/sgx.c | 44 +++++++++++++++++++++++++++++++++++++++
include/hw/i386/sgx-epc.h | 3 +++
monitor/hmp-cmds.c | 1 +
qapi/machine.json | 10 ++++++++-
qemu-options.hx | 4 ++--
9 files changed, 70 insertions(+), 6 deletions(-)
diff --git a/hw/core/numa.c b/hw/core/numa.c
index e6050b22739f44a0c9acbed54599..1aa05dcf425f46eca6ac0468d56a 100644
--- a/hw/core/numa.c
+++ b/hw/core/numa.c
@@ -784,9 +784,8 @@ static void numa_stat_memory_devices(NumaNodeMem node_mem[])
break;
case MEMORY_DEVICE_INFO_KIND_SGX_EPC:
se = value->u.sgx_epc.data;
- /* TODO: once we support numa, assign to right node */
- node_mem[0].node_mem += se->size;
- node_mem[0].node_plugged_mem += se->size;
+ node_mem[se->node].node_mem += se->size;
+ node_mem[se->node].node_plugged_mem = 0;
break;
default:
g_assert_not_reached();
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index a99c6e4fe3fad88da568c9f738d7..8383b83ee36027631356f28aaedd 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -2068,6 +2068,8 @@ build_srat(GArray *table_data, BIOSLinker *linker, MachineState *machine)
nvdimm_build_srat(table_data);
}
+ sgx_epc_build_srat(table_data);
+
/*
* TODO: this part is not in ACPI spec and current linux kernel boots fine
* without these entries. But I recall there were issues the last time I
diff --git a/hw/i386/sgx-epc.c b/hw/i386/sgx-epc.c
index e508827e787b7b7a38cf9785a030..96b2940d75eba983124effc9963f 100644
--- a/hw/i386/sgx-epc.c
+++ b/hw/i386/sgx-epc.c
@@ -21,6 +21,7 @@
static Property sgx_epc_properties[] = {
DEFINE_PROP_UINT64(SGX_EPC_ADDR_PROP, SGXEPCDevice, addr, 0),
+ DEFINE_PROP_UINT32(SGX_EPC_NUMA_NODE_PROP, SGXEPCDevice, node, 0),
DEFINE_PROP_LINK(SGX_EPC_MEMDEV_PROP, SGXEPCDevice, hostmem,
TYPE_MEMORY_BACKEND_EPC, HostMemoryBackendEpc *),
DEFINE_PROP_END_OF_LIST(),
@@ -139,6 +140,8 @@ static void sgx_epc_md_fill_device_info(const MemoryDeviceState *md,
se->memaddr = epc->addr;
se->size = object_property_get_uint(OBJECT(epc), SGX_EPC_SIZE_PROP,
NULL);
+ se->node = object_property_get_uint(OBJECT(epc), SGX_EPC_NUMA_NODE_PROP,
+ NULL);
se->memdev = object_get_canonical_path(OBJECT(epc->hostmem));
info->u.sgx_epc.data = se;
diff --git a/hw/i386/sgx-stub.c b/hw/i386/sgx-stub.c
index c9b379e66519210337f039523593..26833eb233c5b6448831c76c3126 100644
--- a/hw/i386/sgx-stub.c
+++ b/hw/i386/sgx-stub.c
@@ -6,6 +6,10 @@
#include "qapi/error.h"
#include "qapi/qapi-commands-misc-target.h"
+void sgx_epc_build_srat(GArray *table_data)
+{
+}
+
SGXInfo *qmp_query_sgx(Error **errp)
{
error_setg(errp, "SGX support is not compiled in");
diff --git a/hw/i386/sgx.c b/hw/i386/sgx.c
index 8fef3dd8fad4c20a4c9a3169f6e3..d04299904a2315308cbaa0ef7dcf 100644
--- a/hw/i386/sgx.c
+++ b/hw/i386/sgx.c
@@ -23,6 +23,7 @@
#include "sysemu/hw_accel.h"
#include "sysemu/reset.h"
#include <sys/ioctl.h>
+#include "hw/acpi/aml-build.h"
#define SGX_MAX_EPC_SECTIONS 8
#define SGX_CPUID_EPC_INVALID 0x0
@@ -36,6 +37,46 @@
#define RETRY_NUM 2
+static int sgx_epc_device_list(Object *obj, void *opaque)
+{
+ GSList **list = opaque;
+
+ if (object_dynamic_cast(obj, TYPE_SGX_EPC)) {
+ *list = g_slist_append(*list, DEVICE(obj));
+ }
+
+ object_child_foreach(obj, sgx_epc_device_list, opaque);
+ return 0;
+}
+
+static GSList *sgx_epc_get_device_list(void)
+{
+ GSList *list = NULL;
+
+ object_child_foreach(qdev_get_machine(), sgx_epc_device_list, &list);
+ return list;
+}
+
+void sgx_epc_build_srat(GArray *table_data)
+{
+ GSList *device_list = sgx_epc_get_device_list();
+
+ for (; device_list; device_list = device_list->next) {
+ DeviceState *dev = device_list->data;
+ Object *obj = OBJECT(dev);
+ uint64_t addr, size;
+ int node;
+
+ node = object_property_get_uint(obj, SGX_EPC_NUMA_NODE_PROP,
+ &error_abort);
+ addr = object_property_get_uint(obj, SGX_EPC_ADDR_PROP, &error_abort);
+ size = object_property_get_uint(obj, SGX_EPC_SIZE_PROP, &error_abort);
+
+ build_srat_memory(table_data, addr, size, node, MEM_AFFINITY_ENABLED);
+ }
+ g_slist_free(device_list);
+}
+
static uint64_t sgx_calc_section_metric(uint64_t low, uint64_t high)
{
return (low & MAKE_64BIT_MASK(12, 20)) +
@@ -226,6 +267,9 @@ void pc_machine_init_sgx_epc(PCMachineState *pcms)
/* set the memdev link with memory backend */
object_property_parse(obj, SGX_EPC_MEMDEV_PROP, list->value->memdev,
&error_fatal);
+ /* set the numa node property for sgx epc object */
+ object_property_set_uint(obj, SGX_EPC_NUMA_NODE_PROP, list->value->node,
+ &error_fatal);
object_property_set_bool(obj, "realized", true, &error_fatal);
object_unref(obj);
}
diff --git a/include/hw/i386/sgx-epc.h b/include/hw/i386/sgx-epc.h
index a6a65be854f88db0e0c7b5d565a0..581fac389a630d66bc173d389e51 100644
--- a/include/hw/i386/sgx-epc.h
+++ b/include/hw/i386/sgx-epc.h
@@ -25,6 +25,7 @@
#define SGX_EPC_ADDR_PROP "addr"
#define SGX_EPC_SIZE_PROP "size"
#define SGX_EPC_MEMDEV_PROP "memdev"
+#define SGX_EPC_NUMA_NODE_PROP "node"
/**
* SGXEPCDevice:
@@ -38,6 +39,7 @@ typedef struct SGXEPCDevice {
/* public */
uint64_t addr;
+ uint32_t node;
HostMemoryBackendEpc *hostmem;
} SGXEPCDevice;
@@ -56,6 +58,7 @@ typedef struct SGXEPCState {
} SGXEPCState;
bool sgx_epc_get_section(int section_nr, uint64_t *addr, uint64_t *size);
+void sgx_epc_build_srat(GArray *table_data);
static inline uint64_t sgx_epc_above_4g_end(SGXEPCState *sgx_epc)
{
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index 9c91bf93e94cda6baccc53e9c0a3..2669156b284868188392a2da75b2 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -1810,6 +1810,7 @@ void hmp_info_memory_devices(Monitor *mon, const QDict *qdict)
se->id ? se->id : "");
monitor_printf(mon, " memaddr: 0x%" PRIx64 "\n", se->memaddr);
monitor_printf(mon, " size: %" PRIu64 "\n", se->size);
+ monitor_printf(mon, " node: %" PRId64 "\n", se->node);
monitor_printf(mon, " memdev: %s\n", se->memdev);
break;
default:
diff --git a/qapi/machine.json b/qapi/machine.json
index 067e3f53787928d38566e1e26e1c..16e771affcf6d4e085caf7e3d53e 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -1207,12 +1207,15 @@
#
# @memdev: memory backend linked with device
#
+# @node: the numa node
+#
# Since: 6.2
##
{ 'struct': 'SgxEPCDeviceInfo',
'data': { '*id': 'str',
'memaddr': 'size',
'size': 'size',
+ 'node': 'int',
'memdev': 'str'
}
}
@@ -1285,10 +1288,15 @@
#
# @memdev: memory backend linked with device
#
+# @node: the numa node
+#
# Since: 6.2
##
{ 'struct': 'SgxEPC',
- 'data': { 'memdev': 'str' } }
+ 'data': { 'memdev': 'str',
+ 'node': 'int'
+ }
+}
##
# @SgxEPCProperties:
diff --git a/qemu-options.hx b/qemu-options.hx
index ae2c6dbbfc005c526026604d9886..489b58e15110662382136feaf0ae 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -127,11 +127,11 @@ SRST
ERST
DEF("M", HAS_ARG, QEMU_OPTION_M,
- " sgx-epc.0.memdev=memid\n",
+ " sgx-epc.0.memdev=memid,sgx-epc.0.node=numaid\n",
QEMU_ARCH_ALL)
SRST
-``sgx-epc.0.memdev=@var{memid}``
+``sgx-epc.0.memdev=@var{memid},sgx-epc.0.node=@var{numaid}``
Define an SGX EPC section.
ERST

199
numa-Support-SGX-numa-in-the-monitor-and.patch
View File

@ -1,199 +0,0 @@
From: Yang Zhong <yang.zhong@intel.com>
Date: Mon, 1 Nov 2021 12:20:07 -0400
Subject: numa: Support SGX numa in the monitor and Libvirt interfaces
Git-commit: 4755927ae12547c2e7cb22c5fa1b39038c6c11b1
References: bsc#1197807
Add the SGXEPCSection list into SGXInfo to show the multiple
SGX EPC sections detailed info, not the total size like before.
This patch can enable numa support for 'info sgx' command and
QMP interfaces. The new interfaces show each EPC section info
in one numa node. Libvirt can use QMP interface to get the
detailed host SGX EPC capabilities to decide how to allocate
host EPC sections to guest.
(qemu) info sgx
SGX support: enabled
SGX1 support: enabled
SGX2 support: enabled
FLC support: enabled
NUMA node #0: size=67108864
NUMA node #1: size=29360128
The QMP interface show:
(QEMU) query-sgx
{"return": {"sgx": true, "sgx2": true, "sgx1": true, "sections": \
[{"node": 0, "size": 67108864}, {"node": 1, "size": 29360128}], "flc": true}}
(QEMU) query-sgx-capabilities
{"return": {"sgx": true, "sgx2": true, "sgx1": true, "sections": \
[{"node": 0, "size": 17070817280}, {"node": 1, "size": 17079205888}], "flc": true}}
Signed-off-by: Yang Zhong <yang.zhong@intel.com>
Message-Id: <20211101162009.62161-4-yang.zhong@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
hw/i386/sgx.c | 51 +++++++++++++++++++++++++++++++++++--------
qapi/misc-target.json | 19 ++++++++++++++--
2 files changed, 59 insertions(+), 11 deletions(-)
diff --git a/hw/i386/sgx.c b/hw/i386/sgx.c
index d04299904a2315308cbaa0ef7dcf..5de5dd08936e10e8966722f5fac8 100644
--- a/hw/i386/sgx.c
+++ b/hw/i386/sgx.c
@@ -83,11 +83,13 @@ static uint64_t sgx_calc_section_metric(uint64_t low, uint64_t high)
((high & MAKE_64BIT_MASK(0, 20)) << 32);
}
-static uint64_t sgx_calc_host_epc_section_size(void)
+static SGXEPCSectionList *sgx_calc_host_epc_sections(void)
{
+ SGXEPCSectionList *head = NULL, **tail = &head;
+ SGXEPCSection *section;
uint32_t i, type;
uint32_t eax, ebx, ecx, edx;
- uint64_t size = 0;
+ uint32_t j = 0;
for (i = 0; i < SGX_MAX_EPC_SECTIONS; i++) {
host_cpuid(0x12, i + 2, &eax, &ebx, &ecx, &edx);
@@ -101,10 +103,13 @@ static uint64_t sgx_calc_host_epc_section_size(void)
break;
}
- size += sgx_calc_section_metric(ecx, edx);
+ section = g_new0(SGXEPCSection, 1);
+ section->node = j++;
+ section->size = sgx_calc_section_metric(ecx, edx);
+ QAPI_LIST_APPEND(tail, section);
}
- return size;
+ return head;
}
static void sgx_epc_reset(void *opaque)
@@ -168,13 +173,35 @@ SGXInfo *qmp_query_sgx_capabilities(Error **errp)
info->sgx1 = eax & (1U << 0) ? true : false;
info->sgx2 = eax & (1U << 1) ? true : false;
- info->section_size = sgx_calc_host_epc_section_size();
+ info->sections = sgx_calc_host_epc_sections();
close(fd);
return info;
}
+static SGXEPCSectionList *sgx_get_epc_sections_list(void)
+{
+ GSList *device_list = sgx_epc_get_device_list();
+ SGXEPCSectionList *head = NULL, **tail = &head;
+ SGXEPCSection *section;
+
+ for (; device_list; device_list = device_list->next) {
+ DeviceState *dev = device_list->data;
+ Object *obj = OBJECT(dev);
+
+ section = g_new0(SGXEPCSection, 1);
+ section->node = object_property_get_uint(obj, SGX_EPC_NUMA_NODE_PROP,
+ &error_abort);
+ section->size = object_property_get_uint(obj, SGX_EPC_SIZE_PROP,
+ &error_abort);
+ QAPI_LIST_APPEND(tail, section);
+ }
+ g_slist_free(device_list);
+
+ return head;
+}
+
SGXInfo *qmp_query_sgx(Error **errp)
{
SGXInfo *info = NULL;
@@ -193,14 +220,13 @@ SGXInfo *qmp_query_sgx(Error **errp)
return NULL;
}
- SGXEPCState *sgx_epc = &pcms->sgx_epc;
info = g_new0(SGXInfo, 1);
info->sgx = true;
info->sgx1 = true;
info->sgx2 = true;
info->flc = true;
- info->section_size = sgx_epc->size;
+ info->sections = sgx_get_epc_sections_list();
return info;
}
@@ -208,6 +234,7 @@ SGXInfo *qmp_query_sgx(Error **errp)
void hmp_info_sgx(Monitor *mon, const QDict *qdict)
{
Error *err = NULL;
+ SGXEPCSectionList *section_list, *section;
g_autoptr(SGXInfo) info = qmp_query_sgx(&err);
if (err) {
@@ -222,8 +249,14 @@ void hmp_info_sgx(Monitor *mon, const QDict *qdict)
info->sgx2 ? "enabled" : "disabled");
monitor_printf(mon, "FLC support: %s\n",
info->flc ? "enabled" : "disabled");
- monitor_printf(mon, "size: %" PRIu64 "\n",
- info->section_size);
+
+ section_list = info->sections;
+ for (section = section_list; section; section = section->next) {
+ monitor_printf(mon, "NUMA node #%" PRId64 ": ",
+ section->value->node);
+ monitor_printf(mon, "size=%" PRIu64 "\n",
+ section->value->size);
+ }
}
bool sgx_epc_get_section(int section_nr, uint64_t *addr, uint64_t *size)
diff --git a/qapi/misc-target.json b/qapi/misc-target.json
index 5aa2b95b7d4aa09a4b11a9bbabaf..1022aa0184c74a16c8380e310bb3 100644
--- a/qapi/misc-target.json
+++ b/qapi/misc-target.json
@@ -337,6 +337,21 @@
'if': 'TARGET_ARM' }
+##
+# @SGXEPCSection:
+#
+# Information about intel SGX EPC section info
+#
+# @node: the numa node
+#
+# @size: the size of epc section
+#
+# Since: 6.2
+##
+{ 'struct': 'SGXEPCSection',
+ 'data': { 'node': 'int',
+ 'size': 'uint64'}}
+
##
# @SGXInfo:
#
@@ -350,7 +365,7 @@
#
# @flc: true if FLC is supported
#
-# @section-size: The EPC section size for guest
+# @sections: The EPC sections info for guest
#
# Since: 6.2
##
@@ -359,7 +374,7 @@
'sgx1': 'bool',
'sgx2': 'bool',
'flc': 'bool',
- 'section-size': 'uint64'},
+ 'sections': ['SGXEPCSection']},
'if': 'TARGET_I386' }
##

4
pci-fix-overflow-in-snprintf-string-form.patch
View File

@ -71,10 +71,10 @@ Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
1 file changed, 9 insertions(+), 9 deletions(-)
diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index e5993c1ef52b7c9e39faa7de4020..87c419836b3c990ee862f623fd89 100644
index dae9119bfe5fe45a15e31985151e..c69b412434775df1368beb67984c 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -2576,15 +2576,15 @@ static char *pci_dev_fw_name(DeviceState *dev, char *buf, int len)
@@ -2625,15 +2625,15 @@ static char *pci_dev_fw_name(DeviceState *dev, char *buf, int len)
static char *pcibus_get_fw_dev_path(DeviceState *dev)
{
PCIDevice *d = (PCIDevice *)dev;

36
python-aqmp-Fix-negotiation-with-pre-oob.patch
View File

@ -1,36 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Mon, 31 Jan 2022 23:11:31 -0500
Subject: python/aqmp: Fix negotiation with pre-"oob" QEMU
Git-commit: fa73e6e4ca1a93c5bbf9d05fb2a25736ab810b35
QEMU versions prior to the "oob" capability *also* can't accept the
"enable" keyword argument at all. Fix the handshake process with older
QEMU versions.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20220201041134.1237016-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/qmp_client.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/python/qemu/aqmp/qmp_client.py b/python/qemu/aqmp/qmp_client.py
index 8105e29fa8f04297ec9390ec25ea..6b43e1dbbe38eded19fd0115e8bc 100644
--- a/python/qemu/aqmp/qmp_client.py
+++ b/python/qemu/aqmp/qmp_client.py
@@ -292,9 +292,9 @@ class QMPClient(AsyncProtocol[Message], Events):
"""
self.logger.debug("Negotiating capabilities ...")
- arguments: Dict[str, List[str]] = {'enable': []}
+ arguments: Dict[str, List[str]] = {}
if self._greeting and 'oob' in self._greeting.QMP.capabilities:
- arguments['enable'].append('oob')
+ arguments.setdefault('enable', []).append('oob')
msg = self.make_execute_msg('qmp_capabilities', arguments=arguments)
# It's not safe to use execute() here, because the reader/writers

42
python-aqmp-add-SocketAddrT-to-package-r.patch
View File

@ -1,42 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Mon, 10 Jan 2022 18:28:48 -0500
Subject: python/aqmp: add SocketAddrT to package root
Git-commit: 728dcac5e356ce5b948943f21c0c72a1b2d96122
It's a commonly needed definition, it can be re-exported by the root.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/__init__.py | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/python/qemu/aqmp/__init__.py b/python/qemu/aqmp/__init__.py
index 880d5b6fa7f2c966542b12e25571..c6fa2dda58fdf4f1a867e677eadb 100644
--- a/python/qemu/aqmp/__init__.py
+++ b/python/qemu/aqmp/__init__.py
@@ -26,7 +26,12 @@ import logging
from .error import AQMPError
from .events import EventListener
from .message import Message
-from .protocol import ConnectError, Runstate, StateError
+from .protocol import (
+ ConnectError,
+ Runstate,
+ SocketAddrT,
+ StateError,
+)
from .qmp_client import ExecInterruptedError, ExecuteError, QMPClient
@@ -48,4 +53,7 @@ __all__ = (
'ConnectError',
'ExecuteError',
'ExecInterruptedError',
+
+ # Type aliases
+ 'SocketAddrT',
)

64
python-aqmp-add-__del__-method-to-legacy.patch
View File

@ -1,64 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Mon, 10 Jan 2022 18:28:45 -0500
Subject: python/aqmp: add __del__ method to legacy interface
Git-commit: 3bc72e3aed76e0326703db81964b13f1da075cbf
asyncio can complain *very* loudly if you forget to back out of things
gracefully before the garbage collector starts destroying objects that
contain live references to asyncio Tasks.
The usual fix is just to remember to call aqmp.disconnect(), but for the
sake of the legacy wrapper and quick, one-off scripts where a graceful
shutdown is not necessarily of paramount imporance, add a courtesy
cleanup that will trigger prior to seeing screenfuls of confusing
asyncio tracebacks.
Note that we can't *always* save you from yourself; depending on when
the GC runs, you might just seriously be out of luck. The best we can do
in this case is to gently remind you to clean up after yourself.
(Still much better than multiple pages of incomprehensible python
warnings for the crime of forgetting to put your toys away.)
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/legacy.py | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/python/qemu/aqmp/legacy.py b/python/qemu/aqmp/legacy.py
index 9e7b9fb80b95ad5d442fea0dbbab..2ccb136b02c1fcf586507400c056 100644
--- a/python/qemu/aqmp/legacy.py
+++ b/python/qemu/aqmp/legacy.py
@@ -16,6 +16,8 @@ from typing import (
import qemu.qmp
from qemu.qmp import QMPMessage, QMPReturnValue, SocketAddrT
+from .error import AQMPError
+from .protocol import Runstate
from .qmp_client import QMPClient
@@ -136,3 +138,19 @@ class QEMUMonitorProtocol(qemu.qmp.QEMUMonitorProtocol):
def send_fd_scm(self, fd: int) -> None:
self._aqmp.send_fd_scm(fd)
+
+ def __del__(self) -> None:
+ if self._aqmp.runstate == Runstate.IDLE:
+ return
+
+ if not self._aloop.is_running():
+ self.close()
+ else:
+ # Garbage collection ran while the event loop was running.
+ # Nothing we can do about it now, but if we don't raise our
+ # own error, the user will be treated to a lot of traceback
+ # they might not understand.
+ raise AQMPError(
+ "QEMUMonitorProtocol.close()"
+ " was not called before object was garbage collected"
+ )

140
python-aqmp-add-_session_guard.patch
View File

@ -1,140 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Fri, 25 Feb 2022 15:59:39 -0500
Subject: python/aqmp: add _session_guard()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 40196c23939758abc5300e85333e676196e3ba6d
In _new_session, there's a fairly complex except clause that's used to
give semantic errors to callers of accept() and connect(). We need to
create a new two-step replacement for accept(), so factoring out this
piece of logic will be useful.
Bolster the comments and docstring here to try and demystify what's
going on in this fairly delicate piece of Python magic.
(If we were using Python 3.7+, this would be an @asynccontextmanager. We
don't have that very nice piece of magic, however, so this must take an
Awaitable to manage the Exception contexts properly. We pay the price
for platform compatibility.)
Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/protocol.py | 89 +++++++++++++++++++++++++-----------
1 file changed, 62 insertions(+), 27 deletions(-)
diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py
index 33358f5cd72b61bd060b8dea6091..009883f64d011e44dd003e9dcde3 100644
--- a/python/qemu/aqmp/protocol.py
+++ b/python/qemu/aqmp/protocol.py
@@ -317,6 +317,62 @@ class AsyncProtocol(Generic[T]):
# Section: Session machinery
# --------------------------
+ async def _session_guard(self, coro: Awaitable[None], emsg: str) -> None:
+ """
+ Async guard function used to roll back to `IDLE` on any error.
+
+ On any Exception, the state machine will be reset back to
+ `IDLE`. Most Exceptions will be wrapped with `ConnectError`, but
+ `BaseException` events will be left alone (This includes
+ asyncio.CancelledError, even prior to Python 3.8).
+
+ :param error_message:
+ Human-readable string describing what connection phase failed.
+
+ :raise BaseException:
+ When `BaseException` occurs in the guarded block.
+ :raise ConnectError:
+ When any other error is encountered in the guarded block.
+ """
+ # Note: After Python 3.6 support is removed, this should be an
+ # @asynccontextmanager instead of accepting a callback.
+ try:
+ await coro
+ except BaseException as err:
+ self.logger.error("%s: %s", emsg, exception_summary(err))
+ self.logger.debug("%s:\n%s\n", emsg, pretty_traceback())
+ try:
+ # Reset the runstate back to IDLE.
+ await self.disconnect()
+ except:
+ # We don't expect any Exceptions from the disconnect function
+ # here, because we failed to connect in the first place.
+ # The disconnect() function is intended to perform
+ # only cannot-fail cleanup here, but you never know.
+ emsg = (
+ "Unexpected bottom half exception. "
+ "This is a bug in the QMP library. "
+ "Please report it to <qemu-devel@nongnu.org> and "
+ "CC: John Snow <jsnow@redhat.com>."
+ )
+ self.logger.critical("%s:\n%s\n", emsg, pretty_traceback())
+ raise
+
+ # CancelledError is an Exception with special semantic meaning;
+ # We do NOT want to wrap it up under ConnectError.
+ # NB: CancelledError is not a BaseException before Python 3.8
+ if isinstance(err, asyncio.CancelledError):
+ raise
+
+ # Any other kind of error can be treated as some kind of connection
+ # failure broadly. Inspect the 'exc' field to explore the root
+ # cause in greater detail.
+ if isinstance(err, Exception):
+ raise ConnectError(emsg, err) from err
+
+ # Raise BaseExceptions un-wrapped, they're more important.
+ raise
+
@property
def _runstate_event(self) -> asyncio.Event:
# asyncio.Event() objects should not be created prior to entrance into
@@ -371,34 +427,13 @@ class AsyncProtocol(Generic[T]):
"""
assert self.runstate == Runstate.IDLE
- try:
- phase = "connection"
- await self._establish_connection(address, ssl, accept)
-
- phase = "session"
- await self._establish_session()
+ await self._session_guard(
+ self._establish_connection(address, ssl, accept),
+ 'Failed to establish connection')
- except BaseException as err:
- emsg = f"Failed to establish {phase}"
- self.logger.error("%s: %s", emsg, exception_summary(err))
- self.logger.debug("%s:\n%s\n", emsg, pretty_traceback())
- try:
- # Reset from CONNECTING back to IDLE.
- await self.disconnect()
- except:
- emsg = "Unexpected bottom half exception"
- self.logger.critical("%s:\n%s\n", emsg, pretty_traceback())
- raise
-
- # NB: CancelledError is not a BaseException before Python 3.8
- if isinstance(err, asyncio.CancelledError):
- raise
-
- if isinstance(err, Exception):
- raise ConnectError(emsg, err) from err
-
- # Raise BaseExceptions un-wrapped, they're more important.
- raise
+ await self._session_guard(
+ self._establish_session(),
+ 'Failed to establish session')
assert self.runstate == Runstate.RUNNING

135
python-aqmp-add-socket-bind-step-to-lega.patch
View File

@ -1,135 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Mon, 31 Jan 2022 23:11:34 -0500
Subject: python/aqmp: add socket bind step to legacy.py
Git-commit: b0b662bb2b340d63529672b5bdae596a6243c4d0
The synchronous QMP library would bind to the server address during
__init__(). The new library delays this to the accept() call, because
binding occurs inside of the call to start_[unix_]server(), which is an
async method -- so it cannot happen during __init__ anymore.
Python 3.7+ adds the ability to create the server (and thus the bind()
call) and begin the active listening in separate steps, but we don't
have that functionality in 3.6, our current minimum.
Therefore ... Add a temporary workaround that allows the synchronous
version of the client to bind the socket in advance, guaranteeing that
there will be a UNIX socket in the filesystem ready for the QEMU client
to connect to without a race condition.
(Yes, it's a bit ugly. Fixing it more nicely will have to wait until our
minimum Python version is 3.7+.)
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20220201041134.1237016-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/legacy.py | 3 +++
python/qemu/aqmp/protocol.py | 41 +++++++++++++++++++++++++++++++++---
2 files changed, 41 insertions(+), 3 deletions(-)
diff --git a/python/qemu/aqmp/legacy.py b/python/qemu/aqmp/legacy.py
index 0890f95b16875ecb815ed4560bc7..6baa5f3409a6b459c67097d3c2a0 100644
--- a/python/qemu/aqmp/legacy.py
+++ b/python/qemu/aqmp/legacy.py
@@ -56,6 +56,9 @@ class QEMUMonitorProtocol(qemu.qmp.QEMUMonitorProtocol):
self._address = address
self._timeout: Optional[float] = None
+ if server:
+ self._aqmp._bind_hack(address) # pylint: disable=protected-access
+
_T = TypeVar('_T')
def _sync(
diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py
index 50e973c2f2dc9c5fa759380ab3e9..33358f5cd72b61bd060b8dea6091 100644
--- a/python/qemu/aqmp/protocol.py
+++ b/python/qemu/aqmp/protocol.py
@@ -15,6 +15,7 @@ from asyncio import StreamReader, StreamWriter
from enum import Enum
from functools import wraps
import logging
+import socket
from ssl import SSLContext
from typing import (
Any,
@@ -238,6 +239,9 @@ class AsyncProtocol(Generic[T]):
self._runstate = Runstate.IDLE
self._runstate_changed: Optional[asyncio.Event] = None
+ # Workaround for bind()
+ self._sock: Optional[socket.socket] = None
+
def __repr__(self) -> str:
cls_name = type(self).__name__
tokens = []
@@ -427,6 +431,34 @@ class AsyncProtocol(Generic[T]):
else:
await self._do_connect(address, ssl)
+ def _bind_hack(self, address: Union[str, Tuple[str, int]]) -> None:
+ """
+ Used to create a socket in advance of accept().
+
+ This is a workaround to ensure that we can guarantee timing of
+ precisely when a socket exists to avoid a connection attempt
+ bouncing off of nothing.
+
+ Python 3.7+ adds a feature to separate the server creation and
+ listening phases instead, and should be used instead of this
+ hack.
+ """
+ if isinstance(address, tuple):
+ family = socket.AF_INET
+ else:
+ family = socket.AF_UNIX
+
+ sock = socket.socket(family, socket.SOCK_STREAM)
+ sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+
+ try:
+ sock.bind(address)
+ except:
+ sock.close()
+ raise
+
+ self._sock = sock
+
@upper_half
async def _do_accept(self, address: SocketAddrT,
ssl: Optional[SSLContext] = None) -> None:
@@ -464,24 +496,27 @@ class AsyncProtocol(Generic[T]):
if isinstance(address, tuple):
coro = asyncio.start_server(
_client_connected_cb,
- host=address[0],
- port=address[1],
+ host=None if self._sock else address[0],
+ port=None if self._sock else address[1],
ssl=ssl,
backlog=1,
limit=self._limit,
+ sock=self._sock,
)
else:
coro = asyncio.start_unix_server(
_client_connected_cb,
- path=address,
+ path=None if self._sock else address,
ssl=ssl,
backlog=1,
limit=self._limit,
+ sock=self._sock,
)
server = await coro # Starts listening
await connected.wait() # Waits for the callback to fire (and finish)
assert server is None
+ self._sock = None
self.logger.debug("Connection accepted.")

158
python-aqmp-add-start_server-and-accept-.patch
View File

@ -1,158 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Fri, 25 Feb 2022 15:59:46 -0500
Subject: python/aqmp: add start_server() and accept() methods
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 481607c7d35de2bc4d9bec7f4734036fc467f330
Add start_server() and accept() methods that can be used instead of
start_server_and_accept() to allow more fine-grained control over the
incoming connection process.
(Eagle-eyed reviewers will surely notice that it's a bit weird that
"CONNECTING" is a state that's shared between both the start_server()
and connect() states. That's absolutely true, and it's very true that
checking on the presence of _accepted as an indicator of state is a
hack. That's also very certainly true. But ... this keeps client code an
awful lot simpler, as it doesn't have to care exactly *how* the
connection is being made, just that it *is*. Is it worth disrupting that
simplicity in order to provide a better state guard on `accept()`? Hm.)
Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-9-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/protocol.py | 67 +++++++++++++++++++++++++++++++++---
python/tests/protocol.py | 7 ++++
2 files changed, 69 insertions(+), 5 deletions(-)
diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py
index cdbc9cba0d15dea19cc1c60ca3c3..2ecba1455571a35e0e6c565e3641 100644
--- a/python/qemu/aqmp/protocol.py
+++ b/python/qemu/aqmp/protocol.py
@@ -280,6 +280,8 @@ class AsyncProtocol(Generic[T]):
Accept a connection and begin processing message queues.
If this call fails, `runstate` is guaranteed to be set back to `IDLE`.
+ This method is precisely equivalent to calling `start_server()`
+ followed by `accept()`.
:param address:
Address to listen on; UNIX socket path or TCP address/port.
@@ -294,9 +296,62 @@ class AsyncProtocol(Generic[T]):
protocol-level failure occurs while establishing a new
session, the wrapped error may also be an `QMPError`.
"""
+ await self.start_server(address, ssl)
+ await self.accept()
+ assert self.runstate == Runstate.RUNNING
+
+ @upper_half
+ @require(Runstate.IDLE)
+ async def start_server(self, address: SocketAddrT,
+ ssl: Optional[SSLContext] = None) -> None:
+ """
+ Start listening for an incoming connection, but do not wait for a peer.
+
+ This method starts listening for an incoming connection, but
+ does not block waiting for a peer. This call will return
+ immediately after binding and listening on a socket. A later
+ call to `accept()` must be made in order to finalize the
+ incoming connection.
+
+ :param address:
+ Address to listen on; UNIX socket path or TCP address/port.
+ :param ssl: SSL context to use, if any.
+
+ :raise StateError: When the `Runstate` is not `IDLE`.
+ :raise ConnectError:
+ When the server could not start listening on this address.
+
+ This exception will wrap a more concrete one. In most cases,
+ the wrapped exception will be `OSError`.
+ """
await self._session_guard(
self._do_start_server(address, ssl),
'Failed to establish connection')
+ assert self.runstate == Runstate.CONNECTING
+
+ @upper_half
+ @require(Runstate.CONNECTING)
+ async def accept(self) -> None:
+ """
+ Accept an incoming connection and begin processing message queues.
+
+ If this call fails, `runstate` is guaranteed to be set back to `IDLE`.
+
+ :raise StateError: When the `Runstate` is not `CONNECTING`.
+ :raise QMPError: When `start_server()` was not called yet.
+ :raise ConnectError:
+ When a connection or session cannot be established.
+
+ This exception will wrap a more concrete one. In most cases,
+ the wrapped exception will be `OSError` or `EOFError`. If a
+ protocol-level failure occurs while establishing a new
+ session, the wrapped error may also be an `QMPError`.
+ """
+ if self._accepted is None:
+ raise QMPError("Cannot call accept() before start_server().")
+ await self._session_guard(
+ self._do_accept(),
+ 'Failed to establish connection')
await self._session_guard(
self._establish_session(),
'Failed to establish session')
@@ -512,7 +567,12 @@ class AsyncProtocol(Generic[T]):
async def _do_start_server(self, address: SocketAddrT,
ssl: Optional[SSLContext] = None) -> None:
"""
- Acting as the transport server, accept a single connection.
+ Start listening for an incoming connection, but do not wait for a peer.
+
+ This method starts listening for an incoming connection, but does not
+ block waiting for a peer. This call will return immediately after
+ binding and listening to a socket. A later call to accept() must be
+ made in order to finalize the incoming connection.
:param address:
Address to listen on; UNIX socket path or TCP address/port.
@@ -554,10 +614,7 @@ class AsyncProtocol(Generic[T]):
# This will start the server (bind(2), listen(2)). It will also
# call accept(2) if we yield, but we don't block on that here.
self._server = await coro
-
- # Just for this one commit, wait for a peer.
- # This gets split out in the next patch.
- await self._do_accept()
+ self.logger.debug("Server listening on %s", address)
@upper_half
async def _do_accept(self) -> None:
diff --git a/python/tests/protocol.py b/python/tests/protocol.py
index 5e442e1efbd19bf95a95de371060..d6849ad3062081c62b29bf89dd2a 100644
--- a/python/tests/protocol.py
+++ b/python/tests/protocol.py
@@ -43,11 +43,18 @@ class NullProtocol(AsyncProtocol[None]):
async def _do_start_server(self, address, ssl=None):
if self.fake_session:
+ self._accepted = asyncio.Event()
self._set_state(Runstate.CONNECTING)
await asyncio.sleep(0)
else:
await super()._do_start_server(address, ssl)
+ async def _do_accept(self):
+ if self.fake_session:
+ self._accepted = None
+ else:
+ await super()._do_accept()
+
async def _do_connect(self, address, ssl=None):
if self.fake_session:
self._set_state(Runstate.CONNECTING)

134
python-aqmp-copy-type-definitions-from-q.patch
View File

@ -1,134 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Mon, 10 Jan 2022 18:28:47 -0500
Subject: python/aqmp: copy type definitions from qmp
Git-commit: 0e6bfd8b96e407db7b0cb5e8c14cc315a7154f53
Copy the remaining type definitions from QMP into the qemu.aqmp.legacy
module. Now, users that require the legacy interface don't need to
import anything else but qemu.aqmp.legacy wrapper.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/legacy.py | 22 ++++++++++++++++++++--
python/qemu/aqmp/protocol.py | 16 ++++++++++------
2 files changed, 30 insertions(+), 8 deletions(-)
diff --git a/python/qemu/aqmp/legacy.py b/python/qemu/aqmp/legacy.py
index 2ccb136b02c1fcf586507400c056..9431fe933019b1e1c221ea3ab7bb 100644
--- a/python/qemu/aqmp/legacy.py
+++ b/python/qemu/aqmp/legacy.py
@@ -6,7 +6,9 @@ This class pretends to be qemu.qmp.QEMUMonitorProtocol.
import asyncio
from typing import (
+ Any,
Awaitable,
+ Dict,
List,
Optional,
TypeVar,
@@ -14,13 +16,29 @@ from typing import (
)
import qemu.qmp
-from qemu.qmp import QMPMessage, QMPReturnValue, SocketAddrT
from .error import AQMPError
-from .protocol import Runstate
+from .protocol import Runstate, SocketAddrT
from .qmp_client import QMPClient
+#: QMPMessage is an entire QMP message of any kind.
+QMPMessage = Dict[str, Any]
+
+#: QMPReturnValue is the 'return' value of a command.
+QMPReturnValue = object
+
+#: QMPObject is any object in a QMP message.
+QMPObject = Dict[str, object]
+
+# QMPMessage can be outgoing commands or incoming events/returns.
+# QMPReturnValue is usually a dict/json object, but due to QAPI's
+# 'returns-whitelist', it can actually be anything.
+#
+# {'return': {}} is a QMPMessage,
+# {} is the QMPReturnValue.
+
+
# pylint: disable=missing-docstring
diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py
index c4fbe35a0e41c589059ec4fa37a8..5b4f2f0d0a81a0d2902358e9b799 100644
--- a/python/qemu/aqmp/protocol.py
+++ b/python/qemu/aqmp/protocol.py
@@ -46,6 +46,10 @@ T = TypeVar('T')
_U = TypeVar('_U')
_TaskFN = Callable[[], Awaitable[None]] # aka ``async def func() -> None``
+InternetAddrT = Tuple[str, int]
+UnixAddrT = str
+SocketAddrT = Union[UnixAddrT, InternetAddrT]
+
class Runstate(Enum):
"""Protocol session runstate."""
@@ -257,7 +261,7 @@ class AsyncProtocol(Generic[T]):
@upper_half
@require(Runstate.IDLE)
- async def accept(self, address: Union[str, Tuple[str, int]],
+ async def accept(self, address: SocketAddrT,
ssl: Optional[SSLContext] = None) -> None:
"""
Accept a connection and begin processing message queues.
@@ -275,7 +279,7 @@ class AsyncProtocol(Generic[T]):
@upper_half
@require(Runstate.IDLE)
- async def connect(self, address: Union[str, Tuple[str, int]],
+ async def connect(self, address: SocketAddrT,
ssl: Optional[SSLContext] = None) -> None:
"""
Connect to the server and begin processing message queues.
@@ -337,7 +341,7 @@ class AsyncProtocol(Generic[T]):
@upper_half
async def _new_session(self,
- address: Union[str, Tuple[str, int]],
+ address: SocketAddrT,
ssl: Optional[SSLContext] = None,
accept: bool = False) -> None:
"""
@@ -397,7 +401,7 @@ class AsyncProtocol(Generic[T]):
@upper_half
async def _establish_connection(
self,
- address: Union[str, Tuple[str, int]],
+ address: SocketAddrT,
ssl: Optional[SSLContext] = None,
accept: bool = False
) -> None:
@@ -424,7 +428,7 @@ class AsyncProtocol(Generic[T]):
await self._do_connect(address, ssl)
@upper_half
- async def _do_accept(self, address: Union[str, Tuple[str, int]],
+ async def _do_accept(self, address: SocketAddrT,
ssl: Optional[SSLContext] = None) -> None:
"""
Acting as the transport server, accept a single connection.
@@ -482,7 +486,7 @@ class AsyncProtocol(Generic[T]):
self.logger.debug("Connection accepted.")
@upper_half
- async def _do_connect(self, address: Union[str, Tuple[str, int]],
+ async def _do_connect(self, address: SocketAddrT,
ssl: Optional[SSLContext] = None) -> None:
"""
Acting as the transport client, initiate a connection to a server.

132
python-aqmp-drop-_bind_hack.patch
View File

@ -1,132 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Fri, 25 Feb 2022 15:59:48 -0500
Subject: python/aqmp: drop _bind_hack()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 4c1fe7003c9b373acb0791b4356e2285a10365c0
_bind_hack() was a quick fix to allow async QMP to call bind(2) prior to
calling listen(2) and accept(2). This wasn't sufficient to fully address
the race condition present in synchronous clients.
With the race condition in legacy.py fixed (see the previous commit),
there are no longer any users of _bind_hack(). Drop it.
Fixes: b0b662bb2b3
Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-11-jsnow@redhat.com
[Expanded commit message. --js]
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/legacy.py | 2 +-
python/qemu/aqmp/protocol.py | 41 +++---------------------------------
2 files changed, 4 insertions(+), 39 deletions(-)
diff --git a/python/qemu/aqmp/legacy.py b/python/qemu/aqmp/legacy.py
index cb50e60564823fdc0aeeb194c5e3..46026e9fdc6c8859a8c944076c71 100644
--- a/python/qemu/aqmp/legacy.py
+++ b/python/qemu/aqmp/legacy.py
@@ -57,7 +57,7 @@ class QEMUMonitorProtocol(qemu.qmp.QEMUMonitorProtocol):
self._timeout: Optional[float] = None
if server:
- self._sync(self._aqmp.start_server(address))
+ self._sync(self._aqmp.start_server(self._address))
_T = TypeVar('_T')
diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py
index 2ecba1455571a35e0e6c565e3641..36fae57f277826d5cd4026e0d079 100644
--- a/python/qemu/aqmp/protocol.py
+++ b/python/qemu/aqmp/protocol.py
@@ -18,7 +18,6 @@ from asyncio import StreamReader, StreamWriter
from enum import Enum
from functools import wraps
import logging
-import socket
from ssl import SSLContext
from typing import (
Any,
@@ -242,9 +241,6 @@ class AsyncProtocol(Generic[T]):
self._runstate = Runstate.IDLE
self._runstate_changed: Optional[asyncio.Event] = None
- # Workaround for bind()
- self._sock: Optional[socket.socket] = None
-
# Server state for start_server() and _incoming()
self._server: Optional[asyncio.AbstractServer] = None
self._accepted: Optional[asyncio.Event] = None
@@ -535,34 +531,6 @@ class AsyncProtocol(Generic[T]):
self._reader, self._writer = (reader, writer)
self._accepted.set()
- def _bind_hack(self, address: Union[str, Tuple[str, int]]) -> None:
- """
- Used to create a socket in advance of accept().
-
- This is a workaround to ensure that we can guarantee timing of
- precisely when a socket exists to avoid a connection attempt
- bouncing off of nothing.
-
- Python 3.7+ adds a feature to separate the server creation and
- listening phases instead, and should be used instead of this
- hack.
- """
- if isinstance(address, tuple):
- family = socket.AF_INET
- else:
- family = socket.AF_UNIX
-
- sock = socket.socket(family, socket.SOCK_STREAM)
- sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
-
- try:
- sock.bind(address)
- except:
- sock.close()
- raise
-
- self._sock = sock
-
@upper_half
async def _do_start_server(self, address: SocketAddrT,
ssl: Optional[SSLContext] = None) -> None:
@@ -589,21 +557,19 @@ class AsyncProtocol(Generic[T]):
if isinstance(address, tuple):
coro = asyncio.start_server(
self._incoming,
- host=None if self._sock else address[0],
- port=None if self._sock else address[1],
+ host=address[0],
+ port=address[1],
ssl=ssl,
backlog=1,
limit=self._limit,
- sock=self._sock,
)
else:
coro = asyncio.start_unix_server(
self._incoming,
- path=None if self._sock else address,
+ path=address,
ssl=ssl,
backlog=1,
limit=self._limit,
- sock=self._sock,
)
# Allow runstate watchers to witness 'CONNECTING' state; some
@@ -630,7 +596,6 @@ class AsyncProtocol(Generic[T]):
await self._accepted.wait()
assert self._server is None
self._accepted = None
- self._sock = None
self.logger.debug("Connection accepted.")

27
python-aqmp-fix-docstring-typo.patch
View File

@ -1,27 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Mon, 10 Jan 2022 18:28:44 -0500
Subject: python/aqmp: fix docstring typo
Git-commit: dc6877bd2ea04a38700adcea2359d5d20c1082a6
Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/__init__.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/python/qemu/aqmp/__init__.py b/python/qemu/aqmp/__init__.py
index c6fa2dda58fdf4f1a867e677eadb..05f467c1415360169e9c66c8d27e 100644
--- a/python/qemu/aqmp/__init__.py
+++ b/python/qemu/aqmp/__init__.py
@@ -6,7 +6,7 @@ asynchronously with QMP protocol servers, as implemented by QEMU, the
QEMU Guest Agent, and the QEMU Storage Daemon.
`QMPClient` provides the main functionality of this package. All errors
-raised by this library dervive from `AQMPError`, see `aqmp.error` for
+raised by this library derive from `AQMPError`, see `aqmp.error` for
additional detail. See `aqmp.events` for an in-depth tutorial on
managing QMP events.
"""

59
python-aqmp-fix-race-condition-in-legacy.patch
View File

@ -1,59 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Fri, 25 Feb 2022 15:59:47 -0500
Subject: python/aqmp: fix race condition in legacy.py
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 673856f9d889dc50b6a1a7964df960c4f00c7c93
legacy.py provides a synchronous model. iotests frequently uses this
paradigm:
- create QMP client object
- start QEMU process
- await connection from QEMU process
In the switch from sync to async QMP, the QMP client object stopped
calling bind() and listen() during the QMP object creation step, which
creates a race condition if the QEMU process dials in too quickly.
With refactoring out of the way, restore the former behavior of calling
bind() and listen() during __init__() to fix this race condition.
Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-10-jsnow@redhat.com
[Expanded commit message. --js]
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/legacy.py | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/python/qemu/aqmp/legacy.py b/python/qemu/aqmp/legacy.py
index dca1e76ed4994959caf542031363..cb50e60564823fdc0aeeb194c5e3 100644
--- a/python/qemu/aqmp/legacy.py
+++ b/python/qemu/aqmp/legacy.py
@@ -57,7 +57,7 @@ class QEMUMonitorProtocol(qemu.qmp.QEMUMonitorProtocol):
self._timeout: Optional[float] = None
if server:
- self._aqmp._bind_hack(address) # pylint: disable=protected-access
+ self._sync(self._aqmp.start_server(address))
_T = TypeVar('_T')
@@ -90,10 +90,7 @@ class QEMUMonitorProtocol(qemu.qmp.QEMUMonitorProtocol):
self._aqmp.await_greeting = True
self._aqmp.negotiate = True
- self._sync(
- self._aqmp.start_server_and_accept(self._address),
- timeout
- )
+ self._sync(self._aqmp.accept(), timeout)
ret = self._get_greeting()
assert ret is not None

46
python-aqmp-handle-asyncio.TimeoutError-.patch
View File

@ -1,46 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Mon, 10 Jan 2022 18:28:46 -0500
Subject: python/aqmp: handle asyncio.TimeoutError on execute()
Git-commit: 3b5bf136f5798a4ea2c66875d6337ca3d6b79434
This exception can be injected into any await statement. If we are
canceled via timeout, we want to clear the pending execution record on
our way out.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/qmp_client.py | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/python/qemu/aqmp/qmp_client.py b/python/qemu/aqmp/qmp_client.py
index 6b43e1dbbe38eded19fd0115e8bc..45864f288e4fe5a7505c0022ed13 100644
--- a/python/qemu/aqmp/qmp_client.py
+++ b/python/qemu/aqmp/qmp_client.py
@@ -435,7 +435,11 @@ class QMPClient(AsyncProtocol[Message], Events):
msg_id = msg['id']
self._pending[msg_id] = asyncio.Queue(maxsize=1)
- await self._outgoing.put(msg)
+ try:
+ await self._outgoing.put(msg)
+ except:
+ del self._pending[msg_id]
+ raise
return msg_id
@@ -452,9 +456,9 @@ class QMPClient(AsyncProtocol[Message], Events):
was lost, or some other problem.
"""
queue = self._pending[msg_id]
- result = await queue.get()
try:
+ result = await queue.get()
if isinstance(result, ExecInterruptedError):
raise result
return result

102
python-aqmp-refactor-_do_accept-into-two.patch
View File

@ -1,102 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Fri, 25 Feb 2022 15:59:44 -0500
Subject: python/aqmp: refactor _do_accept() into two distinct steps
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 5e9902a030ab832b0b6577764c65ce6a6f874af6
Refactor _do_accept() into _do_start_server() and _do_accept(). As of
this commit, the former calls the latter, but in subsequent commits
they'll be split apart.
(So please forgive the misnomer for _do_start_server(); it will live up
to its name shortly, and the docstring will be updated then too. I'm
just cutting down on some churn.)
Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/protocol.py | 29 ++++++++++++++++++++++++-----
python/tests/protocol.py | 4 ++--
2 files changed, 26 insertions(+), 7 deletions(-)
diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py
index 631bcdaa554f4a104af4e25a3c61..e2bdad542dc0ef451dd200a1d679 100644
--- a/python/qemu/aqmp/protocol.py
+++ b/python/qemu/aqmp/protocol.py
@@ -295,7 +295,7 @@ class AsyncProtocol(Generic[T]):
session, the wrapped error may also be an `QMPError`.
"""
await self._session_guard(
- self._do_accept(address, ssl),
+ self._do_start_server(address, ssl),
'Failed to establish connection')
await self._session_guard(
self._establish_session(),
@@ -509,8 +509,8 @@ class AsyncProtocol(Generic[T]):
self._sock = sock
@upper_half
- async def _do_accept(self, address: SocketAddrT,
- ssl: Optional[SSLContext] = None) -> None:
+ async def _do_start_server(self, address: SocketAddrT,
+ ssl: Optional[SSLContext] = None) -> None:
"""
Acting as the transport server, accept a single connection.
@@ -551,9 +551,28 @@ class AsyncProtocol(Generic[T]):
# otherwise yield.
await asyncio.sleep(0)
- self._server = await coro # Starts listening
- await self._accepted.wait() # Waits for the callback to finish
+ # This will start the server (bind(2), listen(2)). It will also
+ # call accept(2) if we yield, but we don't block on that here.
+ self._server = await coro
+
+ # Just for this one commit, wait for a peer.
+ # This gets split out in the next patch.
+ await self._do_accept()
+
+ @upper_half
+ async def _do_accept(self) -> None:
+ """
+ Wait for and accept an incoming connection.
+
+ Requires that we have not yet accepted an incoming connection
+ from the upper_half, but it's OK if the server is no longer
+ running because the bottom_half has already accepted the
+ connection.
+ """
+ assert self._accepted is not None
+ await self._accepted.wait()
assert self._server is None
+ self._accepted = None
self._sock = None
self.logger.debug("Connection accepted.")
diff --git a/python/tests/protocol.py b/python/tests/protocol.py
index 8dd26c4ed1e0973b8058604c2373..5e442e1efbd19bf95a95de371060 100644
--- a/python/tests/protocol.py
+++ b/python/tests/protocol.py
@@ -41,12 +41,12 @@ class NullProtocol(AsyncProtocol[None]):
self.trigger_input = asyncio.Event()
await super()._establish_session()
- async def _do_accept(self, address, ssl=None):
+ async def _do_start_server(self, address, ssl=None):
if self.fake_session:
self._set_state(Runstate.CONNECTING)
await asyncio.sleep(0)
else:
- await super()._do_accept(address, ssl)
+ await super()._do_start_server(address, ssl)
async def _do_connect(self, address, ssl=None):
if self.fake_session:

231
python-aqmp-remove-_new_session-and-_est.patch
View File

@ -1,231 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Fri, 25 Feb 2022 15:59:41 -0500
Subject: python/aqmp: remove _new_session and _establish_connection
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 68a6cf3ffe3532c0655efbbf5910bd99a1b4a3fa
These two methods attempted to entirely envelop the logic of
establishing a connection to a peer start to finish. However, we need to
break apart the incoming connection step into more granular steps. We
will no longer be able to reasonably constrain the logic inside of these
helper functions.
So, remove them - with _session_guard(), they no longer serve a real
purpose.
Although the public API doesn't change, the internal API does. Now that
there are no intermediary methods between e.g. connect() and
_do_connect(), there's no hook where the runstate is set. As a result,
the test suite changes a little to cope with the new semantics of
_do_accept() and _do_connect().
Lastly, take some pieces of the now-deleted docstrings and move
them up to the public interface level. They were a little more detailed,
and it won't hurt to keep them.
Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/protocol.py | 117 ++++++++++++++---------------------
python/tests/protocol.py | 10 ++-
2 files changed, 53 insertions(+), 74 deletions(-)
diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py
index 73719257e058b7e9e4d8a281bcd9..b7e5e635d886db0efc85f829f42e 100644
--- a/python/qemu/aqmp/protocol.py
+++ b/python/qemu/aqmp/protocol.py
@@ -275,13 +275,25 @@ class AsyncProtocol(Generic[T]):
If this call fails, `runstate` is guaranteed to be set back to `IDLE`.
:param address:
- Address to listen to; UNIX socket path or TCP address/port.
+ Address to listen on; UNIX socket path or TCP address/port.
:param ssl: SSL context to use, if any.
:raise StateError: When the `Runstate` is not `IDLE`.
- :raise ConnectError: If a connection could not be accepted.
+ :raise ConnectError:
+ When a connection or session cannot be established.
+
+ This exception will wrap a more concrete one. In most cases,
+ the wrapped exception will be `OSError` or `EOFError`. If a
+ protocol-level failure occurs while establishing a new
+ session, the wrapped error may also be an `QMPError`.
"""
- await self._new_session(address, ssl, accept=True)
+ await self._session_guard(
+ self._do_accept(address, ssl),
+ 'Failed to establish connection')
+ await self._session_guard(
+ self._establish_session(),
+ 'Failed to establish session')
+ assert self.runstate == Runstate.RUNNING
@upper_half
@require(Runstate.IDLE)
@@ -297,9 +309,21 @@ class AsyncProtocol(Generic[T]):
:param ssl: SSL context to use, if any.
:raise StateError: When the `Runstate` is not `IDLE`.
- :raise ConnectError: If a connection cannot be made to the server.
+ :raise ConnectError:
+ When a connection or session cannot be established.
+
+ This exception will wrap a more concrete one. In most cases,
+ the wrapped exception will be `OSError` or `EOFError`. If a
+ protocol-level failure occurs while establishing a new
+ session, the wrapped error may also be an `QMPError`.
"""
- await self._new_session(address, ssl)
+ await self._session_guard(
+ self._do_connect(address, ssl),
+ 'Failed to establish connection')
+ await self._session_guard(
+ self._establish_session(),
+ 'Failed to establish session')
+ assert self.runstate == Runstate.RUNNING
@upper_half
async def disconnect(self) -> None:
@@ -401,73 +425,6 @@ class AsyncProtocol(Generic[T]):
self._runstate_event.set()
self._runstate_event.clear()
- @upper_half
- async def _new_session(self,
- address: SocketAddrT,
- ssl: Optional[SSLContext] = None,
- accept: bool = False) -> None:
- """
- Establish a new connection and initialize the session.
-
- Connect or accept a new connection, then begin the protocol
- session machinery. If this call fails, `runstate` is guaranteed
- to be set back to `IDLE`.
-
- :param address:
- Address to connect to/listen on;
- UNIX socket path or TCP address/port.
- :param ssl: SSL context to use, if any.
- :param accept: Accept a connection instead of connecting when `True`.
-
- :raise ConnectError:
- When a connection or session cannot be established.
-
- This exception will wrap a more concrete one. In most cases,
- the wrapped exception will be `OSError` or `EOFError`. If a
- protocol-level failure occurs while establishing a new
- session, the wrapped error may also be an `QMPError`.
- """
- assert self.runstate == Runstate.IDLE
-
- await self._session_guard(
- self._establish_connection(address, ssl, accept),
- 'Failed to establish connection')
-
- await self._session_guard(
- self._establish_session(),
- 'Failed to establish session')
-
- assert self.runstate == Runstate.RUNNING
-
- @upper_half
- async def _establish_connection(
- self,
- address: SocketAddrT,
- ssl: Optional[SSLContext] = None,
- accept: bool = False
- ) -> None:
- """
- Establish a new connection.
-
- :param address:
- Address to connect to/listen on;
- UNIX socket path or TCP address/port.
- :param ssl: SSL context to use, if any.
- :param accept: Accept a connection instead of connecting when `True`.
- """
- assert self.runstate == Runstate.IDLE
- self._set_state(Runstate.CONNECTING)
-
- # Allow runstate watchers to witness 'CONNECTING' state; some
- # failures in the streaming layer are synchronous and will not
- # otherwise yield.
- await asyncio.sleep(0)
-
- if accept:
- await self._do_accept(address, ssl)
- else:
- await self._do_connect(address, ssl)
-
def _bind_hack(self, address: Union[str, Tuple[str, int]]) -> None:
"""
Used to create a socket in advance of accept().
@@ -508,6 +465,9 @@ class AsyncProtocol(Generic[T]):
:raise OSError: For stream-related errors.
"""
+ assert self.runstate == Runstate.IDLE
+ self._set_state(Runstate.CONNECTING)
+
self.logger.debug("Awaiting connection on %s ...", address)
connected = asyncio.Event()
server: Optional[asyncio.AbstractServer] = None
@@ -550,6 +510,11 @@ class AsyncProtocol(Generic[T]):
sock=self._sock,
)
+ # Allow runstate watchers to witness 'CONNECTING' state; some
+ # failures in the streaming layer are synchronous and will not
+ # otherwise yield.
+ await asyncio.sleep(0)
+
server = await coro # Starts listening
await connected.wait() # Waits for the callback to fire (and finish)
assert server is None
@@ -569,6 +534,14 @@ class AsyncProtocol(Generic[T]):
:raise OSError: For stream-related errors.
"""
+ assert self.runstate == Runstate.IDLE
+ self._set_state(Runstate.CONNECTING)
+
+ # Allow runstate watchers to witness 'CONNECTING' state; some
+ # failures in the streaming layer are synchronous and will not
+ # otherwise yield.
+ await asyncio.sleep(0)
+
self.logger.debug("Connecting to %s ...", address)
if isinstance(address, tuple):
diff --git a/python/tests/protocol.py b/python/tests/protocol.py
index 354d6559b9d1e3dc3ad29598af3c..8dd26c4ed1e0973b8058604c2373 100644
--- a/python/tests/protocol.py
+++ b/python/tests/protocol.py
@@ -42,11 +42,17 @@ class NullProtocol(AsyncProtocol[None]):
await super()._establish_session()
async def _do_accept(self, address, ssl=None):
- if not self.fake_session:
+ if self.fake_session:
+ self._set_state(Runstate.CONNECTING)
+ await asyncio.sleep(0)
+ else:
await super()._do_accept(address, ssl)
async def _do_connect(self, address, ssl=None):
- if not self.fake_session:
+ if self.fake_session:
+ self._set_state(Runstate.CONNECTING)
+ await asyncio.sleep(0)
+ else:
await super()._do_connect(address, ssl)
async def _do_recv(self) -> None:

221
python-aqmp-rename-AQMPError-to-QMPError.patch
View File

@ -1,221 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Mon, 10 Jan 2022 18:28:49 -0500
Subject: python/aqmp: rename AQMPError to QMPError
Git-commit: 6e7751dc388df6daf425db0e245d4d3a10859803
This is in preparation for renaming qemu.aqmp to qemu.qmp. I should have
done this from this from the very beginning, but it's a convenient time
to make sure this churn is taken care of.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/__init__.py | 6 +++---
python/qemu/aqmp/error.py | 12 ++++++------
python/qemu/aqmp/events.py | 4 ++--
python/qemu/aqmp/legacy.py | 4 ++--
python/qemu/aqmp/protocol.py | 8 ++++----
python/qemu/aqmp/qmp_client.py | 8 ++++----
6 files changed, 21 insertions(+), 21 deletions(-)
diff --git a/python/qemu/aqmp/__init__.py b/python/qemu/aqmp/__init__.py
index 05f467c1415360169e9c66c8d27e..4c22c380790fd0f86402e402628b 100644
--- a/python/qemu/aqmp/__init__.py
+++ b/python/qemu/aqmp/__init__.py
@@ -6,7 +6,7 @@ asynchronously with QMP protocol servers, as implemented by QEMU, the
QEMU Guest Agent, and the QEMU Storage Daemon.
`QMPClient` provides the main functionality of this package. All errors
-raised by this library derive from `AQMPError`, see `aqmp.error` for
+raised by this library derive from `QMPError`, see `aqmp.error` for
additional detail. See `aqmp.events` for an in-depth tutorial on
managing QMP events.
"""
@@ -23,7 +23,7 @@ managing QMP events.
import logging
-from .error import AQMPError
+from .error import QMPError
from .events import EventListener
from .message import Message
from .protocol import (
@@ -48,7 +48,7 @@ __all__ = (
'Runstate',
# Exceptions, most generic to most explicit
- 'AQMPError',
+ 'QMPError',
'StateError',
'ConnectError',
'ExecuteError',
diff --git a/python/qemu/aqmp/error.py b/python/qemu/aqmp/error.py
index 781f49b00877893d7a88f755c67f..24ba4d505410b5fe56390e3d4e02 100644
--- a/python/qemu/aqmp/error.py
+++ b/python/qemu/aqmp/error.py
@@ -1,21 +1,21 @@
"""
-AQMP Error Classes
+QMP Error Classes
This package seeks to provide semantic error classes that are intended
to be used directly by clients when they would like to handle particular
semantic failures (e.g. "failed to connect") without needing to know the
enumeration of possible reasons for that failure.
-AQMPError serves as the ancestor for all exceptions raised by this
+QMPError serves as the ancestor for all exceptions raised by this
package, and is suitable for use in handling semantic errors from this
library. In most cases, individual public methods will attempt to catch
and re-encapsulate various exceptions to provide a semantic
error-handling interface.
-.. admonition:: AQMP Exception Hierarchy Reference
+.. admonition:: QMP Exception Hierarchy Reference
| `Exception`
- | +-- `AQMPError`
+ | +-- `QMPError`
| +-- `ConnectError`
| +-- `StateError`
| +-- `ExecInterruptedError`
@@ -31,11 +31,11 @@ error-handling interface.
"""
-class AQMPError(Exception):
+class QMPError(Exception):
"""Abstract error class for all errors originating from this package."""
-class ProtocolError(AQMPError):
+class ProtocolError(QMPError):
"""
Abstract error class for protocol failures.
diff --git a/python/qemu/aqmp/events.py b/python/qemu/aqmp/events.py
index 5f7150c78d49d9513978103dc9a7..f3d4e2b5e853c39db9e016009db0 100644
--- a/python/qemu/aqmp/events.py
+++ b/python/qemu/aqmp/events.py
@@ -443,7 +443,7 @@ from typing import (
Union,
)
-from .error import AQMPError
+from .error import QMPError
from .message import Message
@@ -451,7 +451,7 @@ EventNames = Union[str, Iterable[str], None]
EventFilter = Callable[[Message], bool]
-class ListenerError(AQMPError):
+class ListenerError(QMPError):
"""
Generic error class for `EventListener`-related problems.
"""
diff --git a/python/qemu/aqmp/legacy.py b/python/qemu/aqmp/legacy.py
index 9431fe933019b1e1c221ea3ab7bb..27df22818a76190e872f08c0852e 100644
--- a/python/qemu/aqmp/legacy.py
+++ b/python/qemu/aqmp/legacy.py
@@ -17,7 +17,7 @@ from typing import (
import qemu.qmp
-from .error import AQMPError
+from .error import QMPError
from .protocol import Runstate, SocketAddrT
from .qmp_client import QMPClient
@@ -168,7 +168,7 @@ class QEMUMonitorProtocol(qemu.qmp.QEMUMonitorProtocol):
# Nothing we can do about it now, but if we don't raise our
# own error, the user will be treated to a lot of traceback
# they might not understand.
- raise AQMPError(
+ raise QMPError(
"QEMUMonitorProtocol.close()"
" was not called before object was garbage collected"
)
diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py
index 5b4f2f0d0a81a0d2902358e9b799..50e973c2f2dc9c5fa759380ab3e9 100644
--- a/python/qemu/aqmp/protocol.py
+++ b/python/qemu/aqmp/protocol.py
@@ -29,7 +29,7 @@ from typing import (
cast,
)
-from .error import AQMPError
+from .error import QMPError
from .util import (
bottom_half,
create_task,
@@ -65,7 +65,7 @@ class Runstate(Enum):
DISCONNECTING = 3
-class ConnectError(AQMPError):
+class ConnectError(QMPError):
"""
Raised when the initial connection process has failed.
@@ -90,7 +90,7 @@ class ConnectError(AQMPError):
return f"{self.error_message}: {cause}"
-class StateError(AQMPError):
+class StateError(QMPError):
"""
An API command (connect, execute, etc) was issued at an inappropriate time.
@@ -363,7 +363,7 @@ class AsyncProtocol(Generic[T]):
This exception will wrap a more concrete one. In most cases,
the wrapped exception will be `OSError` or `EOFError`. If a
protocol-level failure occurs while establishing a new
- session, the wrapped error may also be an `AQMPError`.
+ session, the wrapped error may also be an `QMPError`.
"""
assert self.runstate == Runstate.IDLE
diff --git a/python/qemu/aqmp/qmp_client.py b/python/qemu/aqmp/qmp_client.py
index 45864f288e4fe5a7505c0022ed13..90a8737f03a997f6813ee7cbcaac 100644
--- a/python/qemu/aqmp/qmp_client.py
+++ b/python/qemu/aqmp/qmp_client.py
@@ -20,7 +20,7 @@ from typing import (
cast,
)
-from .error import AQMPError, ProtocolError
+from .error import ProtocolError, QMPError
from .events import Events
from .message import Message
from .models import ErrorResponse, Greeting
@@ -66,7 +66,7 @@ class NegotiationError(_WrappedProtocolError):
"""
-class ExecuteError(AQMPError):
+class ExecuteError(QMPError):
"""
Exception raised by `QMPClient.execute()` on RPC failure.
@@ -87,7 +87,7 @@ class ExecuteError(AQMPError):
self.error_class: str = error_response.error.class_
-class ExecInterruptedError(AQMPError):
+class ExecInterruptedError(QMPError):
"""
Exception raised by `execute()` (et al) when an RPC is interrupted.
@@ -641,7 +641,7 @@ class QMPClient(AsyncProtocol[Message], Events):
sock = self._writer.transport.get_extra_info('socket')
if sock.family != socket.AF_UNIX:
- raise AQMPError("Sending file descriptors requires a UNIX socket.")
+ raise QMPError("Sending file descriptors requires a UNIX socket.")
if not hasattr(sock, 'sendmsg'):
# We need to void the warranty sticker.

163
python-aqmp-rename-accept-to-start_serve.patch
View File

@ -1,163 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Fri, 25 Feb 2022 15:59:40 -0500
Subject: python/aqmp: rename 'accept()' to 'start_server_and_accept()'
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 0ba4e76b23fed77d09be7f56da783ab3f0b2d497
Previously, I had a method named "accept()" that under-the-hood calls
bind(2), listen(2) *and* accept(2). I meant this as a simplification and
counterpart to the one-shot "connect()" method.
This is confusing to readers who expect accept() to mean *just*
accept(2). Since I need to split apart the "accept()" method into
multiple methods anyway (one of which strongly resembling accept(2)), it
feels pertinent to rename this method *now*.
Rename this all-in-one method "start_server_and_accept()" instead.
Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/legacy.py | 2 +-
python/qemu/aqmp/protocol.py | 6 ++++--
python/tests/protocol.py | 24 ++++++++++++------------
3 files changed, 17 insertions(+), 15 deletions(-)
diff --git a/python/qemu/aqmp/legacy.py b/python/qemu/aqmp/legacy.py
index 6baa5f3409a6b459c67097d3c2a0..dca1e76ed4994959caf542031363 100644
--- a/python/qemu/aqmp/legacy.py
+++ b/python/qemu/aqmp/legacy.py
@@ -91,7 +91,7 @@ class QEMUMonitorProtocol(qemu.qmp.QEMUMonitorProtocol):
self._aqmp.negotiate = True
self._sync(
- self._aqmp.accept(self._address),
+ self._aqmp.start_server_and_accept(self._address),
timeout
)
diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py
index 009883f64d011e44dd003e9dcde3..73719257e058b7e9e4d8a281bcd9 100644
--- a/python/qemu/aqmp/protocol.py
+++ b/python/qemu/aqmp/protocol.py
@@ -265,8 +265,10 @@ class AsyncProtocol(Generic[T]):
@upper_half
@require(Runstate.IDLE)
- async def accept(self, address: SocketAddrT,
- ssl: Optional[SSLContext] = None) -> None:
+ async def start_server_and_accept(
+ self, address: SocketAddrT,
+ ssl: Optional[SSLContext] = None
+ ) -> None:
"""
Accept a connection and begin processing message queues.
diff --git a/python/tests/protocol.py b/python/tests/protocol.py
index 5cd7938be35ec61a1d412728f64e..354d6559b9d1e3dc3ad29598af3c 100644
--- a/python/tests/protocol.py
+++ b/python/tests/protocol.py
@@ -413,14 +413,14 @@ class Accept(Connect):
assert family in ('INET', 'UNIX')
if family == 'INET':
- await self.proto.accept(('example.com', 1))
+ await self.proto.start_server_and_accept(('example.com', 1))
elif family == 'UNIX':
- await self.proto.accept('/dev/null')
+ await self.proto.start_server_and_accept('/dev/null')
async def _hanging_connection(self):
with TemporaryDirectory(suffix='.aqmp') as tmpdir:
sock = os.path.join(tmpdir, type(self.proto).__name__ + ".sock")
- await self.proto.accept(sock)
+ await self.proto.start_server_and_accept(sock)
class FakeSession(TestBase):
@@ -449,13 +449,13 @@ class FakeSession(TestBase):
@TestBase.async_test
async def testFakeAccept(self):
"""Test the full state lifecycle (via accept) with a no-op session."""
- await self.proto.accept('/not/a/real/path')
+ await self.proto.start_server_and_accept('/not/a/real/path')
self.assertEqual(self.proto.runstate, Runstate.RUNNING)
@TestBase.async_test
async def testFakeRecv(self):
"""Test receiving a fake/null message."""
- await self.proto.accept('/not/a/real/path')
+ await self.proto.start_server_and_accept('/not/a/real/path')
logname = self.proto.logger.name
with self.assertLogs(logname, level='DEBUG') as context:
@@ -471,7 +471,7 @@ class FakeSession(TestBase):
@TestBase.async_test
async def testFakeSend(self):
"""Test sending a fake/null message."""
- await self.proto.accept('/not/a/real/path')
+ await self.proto.start_server_and_accept('/not/a/real/path')
logname = self.proto.logger.name
with self.assertLogs(logname, level='DEBUG') as context:
@@ -493,7 +493,7 @@ class FakeSession(TestBase):
):
with self.assertRaises(StateError) as context:
if accept:
- await self.proto.accept('/not/a/real/path')
+ await self.proto.start_server_and_accept('/not/a/real/path')
else:
await self.proto.connect('/not/a/real/path')
@@ -504,7 +504,7 @@ class FakeSession(TestBase):
@TestBase.async_test
async def testAcceptRequireRunning(self):
"""Test that accept() cannot be called when Runstate=RUNNING"""
- await self.proto.accept('/not/a/real/path')
+ await self.proto.start_server_and_accept('/not/a/real/path')
await self._prod_session_api(
Runstate.RUNNING,
@@ -515,7 +515,7 @@ class FakeSession(TestBase):
@TestBase.async_test
async def testConnectRequireRunning(self):
"""Test that connect() cannot be called when Runstate=RUNNING"""
- await self.proto.accept('/not/a/real/path')
+ await self.proto.start_server_and_accept('/not/a/real/path')
await self._prod_session_api(
Runstate.RUNNING,
@@ -526,7 +526,7 @@ class FakeSession(TestBase):
@TestBase.async_test
async def testAcceptRequireDisconnecting(self):
"""Test that accept() cannot be called when Runstate=DISCONNECTING"""
- await self.proto.accept('/not/a/real/path')
+ await self.proto.start_server_and_accept('/not/a/real/path')
# Cheat: force a disconnect.
await self.proto.simulate_disconnect()
@@ -541,7 +541,7 @@ class FakeSession(TestBase):
@TestBase.async_test
async def testConnectRequireDisconnecting(self):
"""Test that connect() cannot be called when Runstate=DISCONNECTING"""
- await self.proto.accept('/not/a/real/path')
+ await self.proto.start_server_and_accept('/not/a/real/path')
# Cheat: force a disconnect.
await self.proto.simulate_disconnect()
@@ -576,7 +576,7 @@ class SimpleSession(TestBase):
async def testSmoke(self):
with TemporaryDirectory(suffix='.aqmp') as tmpdir:
sock = os.path.join(tmpdir, type(self.proto).__name__ + ".sock")
- server_task = create_task(self.server.accept(sock))
+ server_task = create_task(self.server.start_server_and_accept(sock))
# give the server a chance to start listening [...]
await asyncio.sleep(0)

149
python-aqmp-split-_client_connected_cb-o.patch
View File

@ -1,149 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Fri, 25 Feb 2022 15:59:42 -0500
Subject: python/aqmp: split _client_connected_cb() out as _incoming()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 830e6fd36e2aef37b158a10dea6c3853ce43b20c
As part of disentangling the monolithic nature of _do_accept(), split
out the incoming callback to prepare for factoring out the "wait for a
peer" step. Namely, this means using an event signal we can wait on from
outside of this method.
Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/protocol.py | 83 +++++++++++++++++++++++++-----------
1 file changed, 58 insertions(+), 25 deletions(-)
diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py
index b7e5e635d886db0efc85f829f42e..56f05b90308c44a86d0978fd2ce6 100644
--- a/python/qemu/aqmp/protocol.py
+++ b/python/qemu/aqmp/protocol.py
@@ -242,6 +242,10 @@ class AsyncProtocol(Generic[T]):
# Workaround for bind()
self._sock: Optional[socket.socket] = None
+ # Server state for start_server() and _incoming()
+ self._server: Optional[asyncio.AbstractServer] = None
+ self._accepted: Optional[asyncio.Event] = None
+
def __repr__(self) -> str:
cls_name = type(self).__name__
tokens = []
@@ -425,6 +429,54 @@ class AsyncProtocol(Generic[T]):
self._runstate_event.set()
self._runstate_event.clear()
+ @bottom_half # However, it does not run from the R/W tasks.
+ async def _stop_server(self) -> None:
+ """
+ Stop listening for / accepting new incoming connections.
+ """
+ if self._server is None:
+ return
+
+ try:
+ self.logger.debug("Stopping server.")
+ self._server.close()
+ await self._server.wait_closed()
+ self.logger.debug("Server stopped.")
+ finally:
+ self._server = None
+
+ @bottom_half # However, it does not run from the R/W tasks.
+ async def _incoming(self,
+ reader: asyncio.StreamReader,
+ writer: asyncio.StreamWriter) -> None:
+ """
+ Accept an incoming connection and signal the upper_half.
+
+ This method does the minimum necessary to accept a single
+ incoming connection. It signals back to the upper_half ASAP so
+ that any errors during session initialization can occur
+ naturally in the caller's stack.
+
+ :param reader: Incoming `asyncio.StreamReader`
+ :param writer: Incoming `asyncio.StreamWriter`
+ """
+ peer = writer.get_extra_info('peername', 'Unknown peer')
+ self.logger.debug("Incoming connection from %s", peer)
+
+ if self._reader or self._writer:
+ # Sadly, we can have more than one pending connection
+ # because of https://bugs.python.org/issue46715
+ # Close any extra connections we don't actually want.
+ self.logger.warning("Extraneous connection inadvertently accepted")
+ writer.close()
+ return
+
+ # A connection has been accepted; stop listening for new ones.
+ assert self._accepted is not None
+ await self._stop_server()
+ self._reader, self._writer = (reader, writer)
+ self._accepted.set()
+
def _bind_hack(self, address: Union[str, Tuple[str, int]]) -> None:
"""
Used to create a socket in advance of accept().
@@ -469,30 +521,11 @@ class AsyncProtocol(Generic[T]):
self._set_state(Runstate.CONNECTING)
self.logger.debug("Awaiting connection on %s ...", address)
- connected = asyncio.Event()
- server: Optional[asyncio.AbstractServer] = None
-
- async def _client_connected_cb(reader: asyncio.StreamReader,
- writer: asyncio.StreamWriter) -> None:
- """Used to accept a single incoming connection, see below."""
- nonlocal server
- nonlocal connected
-
- # A connection has been accepted; stop listening for new ones.
- assert server is not None
- server.close()
- await server.wait_closed()
- server = None
-
- # Register this client as being connected
- self._reader, self._writer = (reader, writer)
-
- # Signal back: We've accepted a client!
- connected.set()
+ self._accepted = asyncio.Event()
if isinstance(address, tuple):
coro = asyncio.start_server(
- _client_connected_cb,
+ self._incoming,
host=None if self._sock else address[0],
port=None if self._sock else address[1],
ssl=ssl,
@@ -502,7 +535,7 @@ class AsyncProtocol(Generic[T]):
)
else:
coro = asyncio.start_unix_server(
- _client_connected_cb,
+ self._incoming,
path=None if self._sock else address,
ssl=ssl,
backlog=1,
@@ -515,9 +548,9 @@ class AsyncProtocol(Generic[T]):
# otherwise yield.
await asyncio.sleep(0)
- server = await coro # Starts listening
- await connected.wait() # Waits for the callback to fire (and finish)
- assert server is None
+ self._server = await coro # Starts listening
+ await self._accepted.wait() # Waits for the callback to finish
+ assert self._server is None
self._sock = None
self.logger.debug("Connection accepted.")

37
python-aqmp-squelch-pylint-warning-for-t.patch
View File

@ -1,37 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Fri, 25 Feb 2022 15:59:43 -0500
Subject: python/aqmp: squelch pylint warning for too many lines
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 1b9c8cb6ce6b5c5911eb715b2d5b0a2671999dde
I would really like to keep this under 1000 lines, I promise. Doesn't
look like it's gonna happen.
Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
(cherry picked from commit 1b9c8cb6ce6b5c5911eb715b2d5b0a2671999dde)
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/protocol.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py
index 56f05b90308c44a86d0978fd2ce6..631bcdaa554f4a104af4e25a3c61 100644
--- a/python/qemu/aqmp/protocol.py
+++ b/python/qemu/aqmp/protocol.py
@@ -10,6 +10,9 @@ In this package, it is used as the implementation for the `QMPClient`
class.
"""
+# It's all the docstrings ... ! It's long for a good reason ^_^;
+# pylint: disable=too-many-lines
+
import asyncio
from asyncio import StreamReader, StreamWriter
from enum import Enum

54
python-aqmp-stop-the-server-during-disco.patch
View File

@ -1,54 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Fri, 25 Feb 2022 15:59:45 -0500
Subject: python/aqmp: stop the server during disconnect()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 32c5abf051d06ff103d9d30eb6a7f3e8bf582334
Before we allow the full separation of starting the server and accepting
new connections, make sure that the disconnect cleans up the server and
its new state, too.
Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-8-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/protocol.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py
index e2bdad542dc0ef451dd200a1d679..cdbc9cba0d15dea19cc1c60ca3c3 100644
--- a/python/qemu/aqmp/protocol.py
+++ b/python/qemu/aqmp/protocol.py
@@ -432,7 +432,7 @@ class AsyncProtocol(Generic[T]):
self._runstate_event.set()
self._runstate_event.clear()
- @bottom_half # However, it does not run from the R/W tasks.
+ @bottom_half
async def _stop_server(self) -> None:
"""
Stop listening for / accepting new incoming connections.
@@ -709,6 +709,7 @@ class AsyncProtocol(Generic[T]):
self._reader = None
self._writer = None
+ self._accepted = None
# NB: _runstate_changed cannot be cleared because we still need it to
# send the final runstate changed event ...!
@@ -732,6 +733,9 @@ class AsyncProtocol(Generic[T]):
def _done(task: Optional['asyncio.Future[Any]']) -> bool:
return task is not None and task.done()
+ # If the server is running, stop it.
+ await self._stop_server()
+
# Are we already in an error pathway? If either of the tasks are
# already done, or if we have no tasks but a reader/writer; we
# must be.

167
python-introduce-qmp-shell-wrap-convenie.patch
View File

@ -1,167 +0,0 @@
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Fri, 28 Jan 2022 16:11:56 +0000
Subject: python: introduce qmp-shell-wrap convenience tool
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 439125293cc9cfb684eb4db23db04199f5f435a2
With the current 'qmp-shell' tool developers must first spawn QEMU with
a suitable -qmp arg and then spawn qmp-shell in a separate terminal
pointing to the right socket.
With 'qmp-shell-wrap' developers can ignore QMP sockets entirely and
just pass the QEMU command and arguments they want. The program will
listen on a UNIX socket and tell QEMU to connect QMP to that.
For example, this:
# qmp-shell-wrap -- qemu-system-x86_64 -display none
Is roughly equivalent of running:
# qemu-system-x86_64 -display none -qmp qmp-shell-1234 &
# qmp-shell qmp-shell-1234
Except that 'qmp-shell-wrap' switches the socket peers around so that
it is the UNIX socket server and QEMU is the socket client. This makes
QEMU reliably go away when qmp-shell-wrap exits, closing the server
socket.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220128161157.36261-2-berrange@redhat.com
[Edited for rebase. --js]
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/qmp_shell.py | 65 ++++++++++++++++++++++++++++++++---
python/setup.cfg | 1 +
scripts/qmp/qmp-shell-wrap | 11 ++++++
3 files changed, 73 insertions(+), 4 deletions(-)
diff --git a/python/qemu/aqmp/qmp_shell.py b/python/qemu/aqmp/qmp_shell.py
index d11bf54b00e5d56616ae57be0006..c60df787fcd50bf8a0109e5f5cd3 100644
--- a/python/qemu/aqmp/qmp_shell.py
+++ b/python/qemu/aqmp/qmp_shell.py
@@ -86,6 +86,7 @@ import logging
import os
import re
import readline
+from subprocess import Popen
import sys
from typing import (
Iterator,
@@ -167,8 +168,10 @@ class QMPShell(QEMUMonitorProtocol):
:param verbose: Echo outgoing QMP messages to console.
"""
def __init__(self, address: SocketAddrT,
- pretty: bool = False, verbose: bool = False):
- super().__init__(address)
+ pretty: bool = False,
+ verbose: bool = False,
+ server: bool = False):
+ super().__init__(address, server=server)
self._greeting: Optional[QMPMessage] = None
self._completer = QMPCompleter()
self._transmode = False
@@ -409,8 +412,10 @@ class HMPShell(QMPShell):
:param verbose: Echo outgoing QMP messages to console.
"""
def __init__(self, address: SocketAddrT,
- pretty: bool = False, verbose: bool = False):
- super().__init__(address, pretty, verbose)
+ pretty: bool = False,
+ verbose: bool = False,
+ server: bool = False):
+ super().__init__(address, pretty, verbose, server)
self._cpu_index = 0
def _cmd_completion(self) -> None:
@@ -533,5 +538,57 @@ def main() -> None:
pass
+def main_wrap() -> None:
+ """
+ qmp-shell-wrap entry point: parse command line arguments and
+ start the REPL.
+ """
+ parser = argparse.ArgumentParser()
+ parser.add_argument('-H', '--hmp', action='store_true',
+ help='Use HMP interface')
+ parser.add_argument('-v', '--verbose', action='store_true',
+ help='Verbose (echo commands sent and received)')
+ parser.add_argument('-p', '--pretty', action='store_true',
+ help='Pretty-print JSON')
+
+ parser.add_argument('command', nargs=argparse.REMAINDER,
+ help='QEMU command line to invoke')
+
+ args = parser.parse_args()
+
+ cmd = args.command
+ if len(cmd) != 0 and cmd[0] == '--':
+ cmd = cmd[1:]
+ if len(cmd) == 0:
+ cmd = ["qemu-system-x86_64"]
+
+ sockpath = "qmp-shell-wrap-%d" % os.getpid()
+ cmd += ["-qmp", "unix:%s" % sockpath]
+
+ shell_class = HMPShell if args.hmp else QMPShell
+
+ try:
+ address = shell_class.parse_address(sockpath)
+ except QMPBadPortError:
+ parser.error(f"Bad port number: {sockpath}")
+ return # pycharm doesn't know error() is noreturn
+
+ try:
+ with shell_class(address, args.pretty, args.verbose, True) as qemu:
+ with Popen(cmd):
+
+ try:
+ qemu.accept()
+ except ConnectError as err:
+ if isinstance(err.exc, OSError):
+ die(f"Couldn't connect to {args.qmp_server}: {err!s}")
+ die(str(err))
+
+ for _ in qemu.repl():
+ pass
+ finally:
+ os.unlink(sockpath)
+
+
if __name__ == '__main__':
main()
diff --git a/python/setup.cfg b/python/setup.cfg
index 0063c757b78638ef651a362af338..bec54e8b0d663191e2b7afbfa350 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -68,6 +68,7 @@ console_scripts =
qom-fuse = qemu.utils.qom_fuse:QOMFuse.entry_point [fuse]
qemu-ga-client = qemu.utils.qemu_ga_client:main
qmp-shell = qemu.aqmp.qmp_shell:main
+ qmp-shell-wrap = qemu.aqmp.qmp_shell:main_wrap
aqmp-tui = qemu.aqmp.aqmp_tui:main [tui]
[flake8]
diff --git a/scripts/qmp/qmp-shell-wrap b/scripts/qmp/qmp-shell-wrap
new file mode 100755
index 0000000000000000000000000000000000000000..9e94da114f5f87588639f6b2cc636391e80c3864
--- /dev/null
+++ b/scripts/qmp/qmp-shell-wrap
@@ -0,0 +1,11 @@
+#!/usr/bin/env python3
+
+import os
+import sys
+
+sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
+from qemu.qmp import qmp_shell
+
+
+if __name__ == '__main__':
+ qmp_shell.main_wrap()

126
python-machine-raise-VMLaunchFailure-exc.patch
View File

@ -1,126 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Mon, 31 Jan 2022 23:11:32 -0500
Subject: python/machine: raise VMLaunchFailure exception from launch()
Git-commit: 50465f94d211beabfbfc80e4f85ec4fad0757570
This allows us to pack in some extra information about the failure,
which guarantees that if the caller did not *intentionally* cause a
failure (by capturing this Exception), some pretty good clues will be
printed at the bottom of the traceback information.
This will help make failures in the event of a non-negative return code
more obvious when they go unhandled; the current behavior in
_post_shutdown() is to print a warning message only in the event of
signal-based terminations (for negative return codes).
(Note: In Python, catching BaseException instead of Exception catches a
broader array of Exception events, including SystemExit and
KeyboardInterrupt. We do not want to "wrap" such exceptions as a
VMLaunchFailure, because that will 'downgrade' the exception from a
BaseException to a regular Exception. We do, however, want to perform
cleanup in either case, so catch on the broadest scope and
wrap-and-re-raise only in the more targeted scope.)
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20220201041134.1237016-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/machine/machine.py | 45 ++++++++++++++++++++---
tests/qemu-iotests/tests/mirror-top-perms | 3 +-
2 files changed, 40 insertions(+), 8 deletions(-)
diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py
index 67ab06ca2b6daa531b7c0ad9f7c2..a5972fab4d2b1fddfa2d5a7db882 100644
--- a/python/qemu/machine/machine.py
+++ b/python/qemu/machine/machine.py
@@ -74,6 +74,35 @@ class QEMUMachineAddDeviceError(QEMUMachineError):
"""
+class VMLaunchFailure(QEMUMachineError):
+ """
+ Exception raised when a VM launch was attempted, but failed.
+ """
+ def __init__(self, exitcode: Optional[int],
+ command: str, output: Optional[str]):
+ super().__init__(exitcode, command, output)
+ self.exitcode = exitcode
+ self.command = command
+ self.output = output
+
+ def __str__(self) -> str:
+ ret = ''
+ if self.__cause__ is not None:
+ name = type(self.__cause__).__name__
+ reason = str(self.__cause__)
+ if reason:
+ ret += f"{name}: {reason}"
+ else:
+ ret += f"{name}"
+ ret += '\n'
+
+ if self.exitcode is not None:
+ ret += f"\tExit code: {self.exitcode}\n"
+ ret += f"\tCommand: {self.command}\n"
+ ret += f"\tOutput: {self.output}\n"
+ return ret
+
+
class AbnormalShutdown(QEMUMachineError):
"""
Exception raised when a graceful shutdown was requested, but not performed.
@@ -397,7 +426,7 @@ class QEMUMachine:
try:
self._launch()
- except:
+ except BaseException as exc:
# We may have launched the process but it may
# have exited before we could connect via QMP.
# Assume the VM didn't launch or is exiting.
@@ -408,11 +437,15 @@ class QEMUMachine:
else:
self._post_shutdown()
- LOG.debug('Error launching VM')
- if self._qemu_full_args:
- LOG.debug('Command: %r', ' '.join(self._qemu_full_args))
- if self._iolog:
- LOG.debug('Output: %r', self._iolog)
+ if isinstance(exc, Exception):
+ raise VMLaunchFailure(
+ exitcode=self.exitcode(),
+ command=' '.join(self._qemu_full_args),
+ output=self._iolog
+ ) from exc
+
+ # Don't wrap 'BaseException'; doing so would downgrade
+ # that exception. However, we still want to clean up.
raise
def _launch(self) -> None:
diff --git a/tests/qemu-iotests/tests/mirror-top-perms b/tests/qemu-iotests/tests/mirror-top-perms
index 0a51a613f39764b2b3ab3fa460ef..b5849978c4158c35e18480186ea2 100755
--- a/tests/qemu-iotests/tests/mirror-top-perms
+++ b/tests/qemu-iotests/tests/mirror-top-perms
@@ -21,7 +21,6 @@
import os
-from qemu.aqmp import ConnectError
from qemu.machine import machine
from qemu.qmp import QMPConnectError
@@ -107,7 +106,7 @@ class TestMirrorTopPerms(iotests.QMPTestCase):
self.vm_b.launch()
print('ERROR: VM B launched successfully, '
'this should not have happened')
- except (QMPConnectError, ConnectError):
+ except (QMPConnectError, machine.VMLaunchFailure):
assert 'Is another process using the image' in self.vm_b.get_log()
result = self.vm.qmp('block-job-cancel',

1143
python-move-qmp-shell-under-the-AQMP-pac.patch
View File

File diff suppressed because it is too large Load Diff

2153
python-move-qmp-utilities-to-python-qemu.patch
View File

File diff suppressed because it is too large Load Diff

121
python-qmp-switch-qmp-shell-to-AQMP.patch
View File

@ -1,121 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Mon, 10 Jan 2022 18:28:53 -0500
Subject: python/qmp: switch qmp-shell to AQMP
Git-commit: f3efd12930f34b9724e15d8fd2ff56a97b67219d
We have a replacement for async QMP, but it doesn't have feature parity
yet. For now, then, port the old tool onto the new backend.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/legacy.py | 3 +++
python/qemu/qmp/qmp_shell.py | 31 +++++++++++++++++--------------
2 files changed, 20 insertions(+), 14 deletions(-)
diff --git a/python/qemu/aqmp/legacy.py b/python/qemu/aqmp/legacy.py
index 27df22818a76190e872f08c0852e..0890f95b16875ecb815ed4560bc7 100644
--- a/python/qemu/aqmp/legacy.py
+++ b/python/qemu/aqmp/legacy.py
@@ -22,6 +22,9 @@ from .protocol import Runstate, SocketAddrT
from .qmp_client import QMPClient
+# (Temporarily) Re-export QMPBadPortError
+QMPBadPortError = qemu.qmp.QMPBadPortError
+
#: QMPMessage is an entire QMP message of any kind.
QMPMessage = Dict[str, Any]
diff --git a/python/qemu/qmp/qmp_shell.py b/python/qemu/qmp/qmp_shell.py
index e7d7eb18f19cae7ac185b333013e..d11bf54b00e5d56616ae57be0006 100644
--- a/python/qemu/qmp/qmp_shell.py
+++ b/python/qemu/qmp/qmp_shell.py
@@ -95,8 +95,13 @@ from typing import (
Sequence,
)
-from qemu import qmp
-from qemu.qmp import QMPMessage
+from qemu.aqmp import ConnectError, QMPError, SocketAddrT
+from qemu.aqmp.legacy import (
+ QEMUMonitorProtocol,
+ QMPBadPortError,
+ QMPMessage,
+ QMPObject,
+)
LOG = logging.getLogger(__name__)
@@ -125,7 +130,7 @@ class QMPCompleter:
return None
-class QMPShellError(qmp.QMPError):
+class QMPShellError(QMPError):
"""
QMP Shell Base error class.
"""
@@ -153,7 +158,7 @@ class FuzzyJSON(ast.NodeTransformer):
return node
-class QMPShell(qmp.QEMUMonitorProtocol):
+class QMPShell(QEMUMonitorProtocol):
"""
QMPShell provides a basic readline-based QMP shell.
@@ -161,7 +166,7 @@ class QMPShell(qmp.QEMUMonitorProtocol):
:param pretty: Pretty-print QMP messages.
:param verbose: Echo outgoing QMP messages to console.
"""
- def __init__(self, address: qmp.SocketAddrT,
+ def __init__(self, address: SocketAddrT,
pretty: bool = False, verbose: bool = False):
super().__init__(address)
self._greeting: Optional[QMPMessage] = None
@@ -237,7 +242,7 @@ class QMPShell(qmp.QEMUMonitorProtocol):
def _cli_expr(self,
tokens: Sequence[str],
- parent: qmp.QMPObject) -> None:
+ parent: QMPObject) -> None:
for arg in tokens:
(key, sep, val) = arg.partition('=')
if sep != '=':
@@ -403,7 +408,7 @@ class HMPShell(QMPShell):
:param pretty: Pretty-print QMP messages.
:param verbose: Echo outgoing QMP messages to console.
"""
- def __init__(self, address: qmp.SocketAddrT,
+ def __init__(self, address: SocketAddrT,
pretty: bool = False, verbose: bool = False):
super().__init__(address, pretty, verbose)
self._cpu_index = 0
@@ -512,19 +517,17 @@ def main() -> None:
try:
address = shell_class.parse_address(args.qmp_server)
- except qmp.QMPBadPortError:
+ except QMPBadPortError:
parser.error(f"Bad port number: {args.qmp_server}")
return # pycharm doesn't know error() is noreturn
with shell_class(address, args.pretty, args.verbose) as qemu:
try:
qemu.connect(negotiate=not args.skip_negotiation)
- except qmp.QMPConnectError:
- die("Didn't get QMP greeting message")
- except qmp.QMPCapabilitiesError:
- die("Couldn't negotiate capabilities")
- except OSError as err:
- die(f"Couldn't connect to {args.qmp_server}: {err!s}")
+ except ConnectError as err:
+ if isinstance(err.exc, OSError):
+ die(f"Couldn't connect to {args.qmp_server}: {err!s}")
+ die(str(err))
for _ in qemu.repl():
pass

184
python-support-recording-QMP-session-to-.patch
View File

@ -1,184 +0,0 @@
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Fri, 28 Jan 2022 16:11:57 +0000
Subject: python: support recording QMP session to a file
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Git-commit: 5c66d7d8de9a00460199669d26cd83fba135bee5
When running QMP commands with very large response payloads, it is often
not easy to spot the info you want. If we can save the response to a
file then tools like 'grep' or 'jq' can be used to extract information.
For convenience of processing, we merge the QMP command and response
dictionaries together:
{
"arguments": {},
"execute": "query-kvm",
"return": {
"enabled": false,
"present": true
}
}
Example usage
$ ./scripts/qmp/qmp-shell-wrap -l q.log -p -- ./build/qemu-system-x86_64 -display none
Welcome to the QMP low-level shell!
Connected
(QEMU) query-kvm
{
"return": {
"enabled": false,
"present": true
}
}
(QEMU) query-mice
{
"return": [
{
"absolute": false,
"current": true,
"index": 2,
"name": "QEMU PS/2 Mouse"
}
]
}
$ jq --slurp '. | to_entries[] | select(.value.execute == "query-kvm") |
.value.return.enabled' < q.log
false
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220128161157.36261-3-berrange@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/qemu/aqmp/qmp_shell.py | 29 ++++++++++++++++++++++-------
python/setup.cfg | 3 +++
2 files changed, 25 insertions(+), 7 deletions(-)
diff --git a/python/qemu/aqmp/qmp_shell.py b/python/qemu/aqmp/qmp_shell.py
index c60df787fcd50bf8a0109e5f5cd3..35691494d0a88070bcf1ad691699 100644
--- a/python/qemu/aqmp/qmp_shell.py
+++ b/python/qemu/aqmp/qmp_shell.py
@@ -89,6 +89,7 @@ import readline
from subprocess import Popen
import sys
from typing import (
+ IO,
Iterator,
List,
NoReturn,
@@ -170,7 +171,8 @@ class QMPShell(QEMUMonitorProtocol):
def __init__(self, address: SocketAddrT,
pretty: bool = False,
verbose: bool = False,
- server: bool = False):
+ server: bool = False,
+ logfile: Optional[str] = None):
super().__init__(address, server=server)
self._greeting: Optional[QMPMessage] = None
self._completer = QMPCompleter()
@@ -180,6 +182,10 @@ class QMPShell(QEMUMonitorProtocol):
'.qmp-shell_history')
self.pretty = pretty
self.verbose = verbose
+ self.logfile = None
+
+ if logfile is not None:
+ self.logfile = open(logfile, "w", encoding='utf-8')
def close(self) -> None:
# Hook into context manager of parent to save shell history.
@@ -320,11 +326,11 @@ class QMPShell(QEMUMonitorProtocol):
self._cli_expr(cmdargs[1:], qmpcmd['arguments'])
return qmpcmd
- def _print(self, qmp_message: object) -> None:
+ def _print(self, qmp_message: object, fh: IO[str] = sys.stdout) -> None:
jsobj = json.dumps(qmp_message,
indent=4 if self.pretty else None,
sort_keys=self.pretty)
- print(str(jsobj))
+ print(str(jsobj), file=fh)
def _execute_cmd(self, cmdline: str) -> bool:
try:
@@ -347,6 +353,9 @@ class QMPShell(QEMUMonitorProtocol):
print('Disconnected')
return False
self._print(resp)
+ if self.logfile is not None:
+ cmd = {**qmpcmd, **resp}
+ self._print(cmd, fh=self.logfile)
return True
def connect(self, negotiate: bool = True) -> None:
@@ -414,8 +423,9 @@ class HMPShell(QMPShell):
def __init__(self, address: SocketAddrT,
pretty: bool = False,
verbose: bool = False,
- server: bool = False):
- super().__init__(address, pretty, verbose, server)
+ server: bool = False,
+ logfile: Optional[str] = None):
+ super().__init__(address, pretty, verbose, server, logfile)
self._cpu_index = 0
def _cmd_completion(self) -> None:
@@ -508,6 +518,8 @@ def main() -> None:
help='Verbose (echo commands sent and received)')
parser.add_argument('-p', '--pretty', action='store_true',
help='Pretty-print JSON')
+ parser.add_argument('-l', '--logfile',
+ help='Save log of all QMP messages to PATH')
default_server = os.environ.get('QMP_SOCKET')
parser.add_argument('qmp_server', action='store',
@@ -526,7 +538,7 @@ def main() -> None:
parser.error(f"Bad port number: {args.qmp_server}")
return # pycharm doesn't know error() is noreturn
- with shell_class(address, args.pretty, args.verbose) as qemu:
+ with shell_class(address, args.pretty, args.verbose, args.logfile) as qemu:
try:
qemu.connect(negotiate=not args.skip_negotiation)
except ConnectError as err:
@@ -550,6 +562,8 @@ def main_wrap() -> None:
help='Verbose (echo commands sent and received)')
parser.add_argument('-p', '--pretty', action='store_true',
help='Pretty-print JSON')
+ parser.add_argument('-l', '--logfile',
+ help='Save log of all QMP messages to PATH')
parser.add_argument('command', nargs=argparse.REMAINDER,
help='QEMU command line to invoke')
@@ -574,7 +588,8 @@ def main_wrap() -> None:
return # pycharm doesn't know error() is noreturn
try:
- with shell_class(address, args.pretty, args.verbose, True) as qemu:
+ with shell_class(address, args.pretty, args.verbose,
+ True, args.logfile) as qemu:
with Popen(cmd):
try:
diff --git a/python/setup.cfg b/python/setup.cfg
index bec54e8b0d663191e2b7afbfa350..241f243e8b94417f9b032c41576b 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -114,7 +114,10 @@ ignore_missing_imports = True
# no Warning level messages displayed, use "--disable=all --enable=classes
# --disable=W".
disable=consider-using-f-string,
+ consider-using-with,
+ too-many-arguments,
too-many-function-args, # mypy handles this with less false positives.
+ too-many-instance-attributes,
no-member, # mypy also handles this better.
[pylint.basic]

232
python-upgrade-mypy-to-0.780.patch
View File

@ -1,232 +0,0 @@
From: John Snow <jsnow@redhat.com>
Date: Mon, 31 Jan 2022 23:11:33 -0500
Subject: python: upgrade mypy to 0.780
Git-commit: 74a1505d279897d2a448c876820a33cbe1f0f22e
We need a slightly newer version of mypy in order to use some features
of the asyncio server functions in the next commit.
(Note: pipenv is not really suited to upgrading individual packages; I
need to replace this tool with something better for the task. For now,
the miscellaneous updates not related to the mypy upgrade are simply
beyond my control. It's on my list to take care of soon.)
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20220201041134.1237016-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
python/Pipfile.lock | 66 ++++++++++++++++++++++++++-------------------
python/setup.cfg | 2 +-
2 files changed, 40 insertions(+), 28 deletions(-)
diff --git a/python/Pipfile.lock b/python/Pipfile.lock
index d2a7dbd88be19fd6db0baa083d8a..ce46404ce0840c693d3c982674ac 100644
--- a/python/Pipfile.lock
+++ b/python/Pipfile.lock
@@ -1,7 +1,7 @@
{
"_meta": {
"hash": {
- "sha256": "784b327272db32403d5a488507853b5afba850ba26a5948e5b6a90c1baef2d9c"
+ "sha256": "f1a25654d884a5b450e38d78b1f2e3ebb9073e421cc4358d4bbb83ac251a5670"
},
"pipfile-spec": 6,
"requires": {
@@ -34,7 +34,7 @@
"sha256:09bdb456e02564731f8b5957cdd0c98a7f01d2db5e90eb1d794c353c28bfd705",
"sha256:6a8a51f64dae307f6e0c9db752b66a7951e282389d8362cc1d39a56f3feeb31d"
],
- "markers": "python_version ~= '3.6'",
+ "index": "pypi",
"version": "==2.6.0"
},
"avocado-framework": {
@@ -50,6 +50,7 @@
"sha256:106fef6dc37dd8c0e2c0a60d3fca3e77460a48907f335fa28420463a6f799736",
"sha256:23e223426b28491b1ced97dc3bbe183027419dfc7982b4fa2f05d5f3ff10711c"
],
+ "index": "pypi",
"version": "==0.3.2"
},
"filelock": {
@@ -57,6 +58,7 @@
"sha256:18d82244ee114f543149c66a6e0c14e9c4f8a1044b5cdaadd0f82159d6a6ff59",
"sha256:929b7d63ec5b7d6b71b0fa5ac14e030b3f70b75747cef1b10da9b879fef15836"
],
+ "index": "pypi",
"version": "==3.0.12"
},
"flake8": {
@@ -88,7 +90,7 @@
"sha256:54161657e8ffc76596c4ede7080ca68cb02962a2e074a2586b695a93a925d36e",
"sha256:e962bff7440364183203d179d7ae9ad90cb1f2b74dcb84300e88ecc42dca3351"
],
- "markers": "python_version < '3.7'",
+ "index": "pypi",
"version": "==5.1.4"
},
"isort": {
@@ -124,7 +126,7 @@
"sha256:ed361bb83436f117f9917d282a456f9e5009ea12fd6de8742d1a4752c3017e93",
"sha256:f5144c75445ae3ca2057faac03fda5a902eff196702b0a24daf1d6ce0650514b"
],
- "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5'",
+ "index": "pypi",
"version": "==1.6.0"
},
"mccabe": {
@@ -136,23 +138,23 @@
},
"mypy": {
"hashes": [
- "sha256:15b948e1302682e3682f11f50208b726a246ab4e6c1b39f9264a8796bb416aa2",
- "sha256:219a3116ecd015f8dca7b5d2c366c973509dfb9a8fc97ef044a36e3da66144a1",
- "sha256:3b1fc683fb204c6b4403a1ef23f0b1fac8e4477091585e0c8c54cbdf7d7bb164",
- "sha256:3beff56b453b6ef94ecb2996bea101a08f1f8a9771d3cbf4988a61e4d9973761",
- "sha256:7687f6455ec3ed7649d1ae574136835a4272b65b3ddcf01ab8704ac65616c5ce",
- "sha256:7ec45a70d40ede1ec7ad7f95b3c94c9cf4c186a32f6bacb1795b60abd2f9ef27",
- "sha256:86c857510a9b7c3104cf4cde1568f4921762c8f9842e987bc03ed4f160925754",
- "sha256:8a627507ef9b307b46a1fea9513d5c98680ba09591253082b4c48697ba05a4ae",
- "sha256:8dfb69fbf9f3aeed18afffb15e319ca7f8da9642336348ddd6cab2713ddcf8f9",
- "sha256:a34b577cdf6313bf24755f7a0e3f3c326d5c1f4fe7422d1d06498eb25ad0c600",
- "sha256:a8ffcd53cb5dfc131850851cc09f1c44689c2812d0beb954d8138d4f5fc17f65",
- "sha256:b90928f2d9eb2f33162405f32dde9f6dcead63a0971ca8a1b50eb4ca3e35ceb8",
- "sha256:c56ffe22faa2e51054c5f7a3bc70a370939c2ed4de308c690e7949230c995913",
- "sha256:f91c7ae919bbc3f96cd5e5b2e786b2b108343d1d7972ea130f7de27fdd547cf3"
+ "sha256:00cb1964a7476e871d6108341ac9c1a857d6bd20bf5877f4773ac5e9d92cd3cd",
+ "sha256:127de5a9b817a03a98c5ae8a0c46a20dc44442af6dcfa2ae7f96cb519b312efa",
+ "sha256:1f3976a945ad7f0a0727aafdc5651c2d3278e3c88dee94e2bf75cd3386b7b2f4",
+ "sha256:2f8c098f12b402c19b735aec724cc9105cc1a9eea405d08814eb4b14a6fb1a41",
+ "sha256:4ef13b619a289aa025f2273e05e755f8049bb4eaba6d703a425de37d495d178d",
+ "sha256:5d142f219bf8c7894dfa79ebfb7d352c4c63a325e75f10dfb4c3db9417dcd135",
+ "sha256:62eb5dd4ea86bda8ce386f26684f7f26e4bfe6283c9f2b6ca6d17faf704dcfad",
+ "sha256:64c36eb0936d0bfb7d8da49f92c18e312ad2e3ed46e5548ae4ca997b0d33bd59",
+ "sha256:75eed74d2faf2759f79c5f56f17388defd2fc994222312ec54ee921e37b31ad4",
+ "sha256:974bebe3699b9b46278a7f076635d219183da26e1a675c1f8243a69221758273",
+ "sha256:a5e5bb12b7982b179af513dddb06fca12285f0316d74f3964078acbfcf4c68f2",
+ "sha256:d31291df31bafb997952dc0a17ebb2737f802c754aed31dd155a8bfe75112c57",
+ "sha256:d3b4941de44341227ece1caaf5b08b23e42ad4eeb8b603219afb11e9d4cfb437",
+ "sha256:eadb865126da4e3c4c95bdb47fe1bb087a3e3ea14d39a3b13224b8a4d9f9a102"
],
"index": "pypi",
- "version": "==0.770"
+ "version": "==0.780"
},
"mypy-extensions": {
"hashes": [
@@ -166,7 +168,7 @@
"sha256:5b327ac1320dc863dca72f4514ecc086f31186744b84a230374cc1fd776feae5",
"sha256:67714da7f7bc052e064859c05c595155bd1ee9f69f76557e21f051443c20947a"
],
- "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
+ "index": "pypi",
"version": "==20.9"
},
"pluggy": {
@@ -174,7 +176,7 @@
"sha256:15b2acde666561e1298d71b523007ed7364de07029219b604cf808bfa1c765b0",
"sha256:966c145cd83c96502c3c3868f50408687b38434af77734af1e9ca461a4081d2d"
],
- "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
+ "index": "pypi",
"version": "==0.13.1"
},
"py": {
@@ -182,7 +184,7 @@
"sha256:21b81bda15b66ef5e1a777a21c4dcd9c20ad3efd0b3f817e7a809035269e1bd3",
"sha256:3b80836aa6d1feeaa108e046da6423ab8f6ceda6468545ae8d02d9d58d18818a"
],
- "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
+ "index": "pypi",
"version": "==1.10.0"
},
"pycodestyle": {
@@ -205,7 +207,7 @@
"sha256:a18f47b506a429f6f4b9df81bb02beab9ca21d0a5fee38ed15aef65f0545519f",
"sha256:d66e804411278594d764fc69ec36ec13d9ae9147193a1740cd34d272ca383b8e"
],
- "markers": "python_version >= '3.5'",
+ "index": "pypi",
"version": "==2.9.0"
},
"pylint": {
@@ -221,13 +223,21 @@
"sha256:c203ec8783bf771a155b207279b9bccb8dea02d8f0c9e5f8ead507bc3246ecc1",
"sha256:ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b"
],
- "markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'",
+ "index": "pypi",
"version": "==2.4.7"
},
"qemu": {
"editable": true,
"path": "."
},
+ "setuptools": {
+ "hashes": [
+ "sha256:22c7348c6d2976a52632c67f7ab0cdf40147db7789f9aed18734643fe9cf3373",
+ "sha256:4ce92f1e1f8f01233ee9952c04f6b81d1e02939d6e1b488428154974a4d0783e"
+ ],
+ "markers": "python_version >= '3.6'",
+ "version": "==59.6.0"
+ },
"six": {
"hashes": [
"sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926",
@@ -294,19 +304,21 @@
"sha256:50b6f157849174217d0656f99dc82fe932884fb250826c18350e159ec6cdf342",
"sha256:779383f6086d90c99ae41cf0ff39aac8a7937a9283ce0a414e5dd782f4c94a84"
],
- "markers": "python_version < '3.8'",
+ "index": "pypi",
"version": "==3.10.0.0"
},
"urwid": {
"hashes": [
"sha256:588bee9c1cb208d0906a9f73c613d2bd32c3ed3702012f51efe318a3f2127eae"
],
+ "index": "pypi",
"version": "==2.1.2"
},
"urwid-readline": {
"hashes": [
"sha256:018020cbc864bb5ed87be17dc26b069eae2755cb29f3a9c569aac3bded1efaf4"
],
+ "index": "pypi",
"version": "==0.13"
},
"virtualenv": {
@@ -314,7 +326,7 @@
"sha256:14fdf849f80dbb29a4eb6caa9875d476ee2a5cf76a5f5415fa2f1606010ab467",
"sha256:2b0126166ea7c9c3661f5b8e06773d28f83322de7a3ff7d06f0aed18c9de6a76"
],
- "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
+ "index": "pypi",
"version": "==20.4.7"
},
"wrapt": {
@@ -328,7 +340,7 @@
"sha256:3607921face881ba3e026887d8150cca609d517579abe052ac81fc5aeffdbd76",
"sha256:51cb66cc54621609dd593d1787f286ee42a5c0adbb4b29abea5a63edc3e03098"
],
- "markers": "python_version < '3.10'",
+ "index": "pypi",
"version": "==3.4.1"
}
}
diff --git a/python/setup.cfg b/python/setup.cfg
index 417e937839b85eecd752b29ad7df..4f4f20571f304507e20ce16cee66 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -41,7 +41,7 @@ devel =
flake8 >= 3.6.0
fusepy >= 2.0.4
isort >= 5.1.2
- mypy >= 0.770
+ mypy >= 0.780
pylint >= 2.8.0
tox >= 3.18.0
urwid >= 2.1.2

108
qcow2-simple-case-support-for-downgradin.patch
View File

@ -1,108 +0,0 @@
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Thu, 23 Dec 2021 17:01:37 +0100
Subject: qcow2: simple case support for downgrading of qcow2 images with zstd
Git-commit: 083c24561a1f52829b5b31a0fb2f7c77efb979c0
If image doesn't have any compressed cluster we can easily switch to
zlib compression, which may allow to downgrade the image.
That's mostly needed to support IMGOPTS='compression_type=zstd' in some
iotests which do qcow2 downgrade.
While being here also fix checkpatch complain against '#' in printf
formatting.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20211223160144.1097696-13-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Li Zhang <lizhang@suse.de>
---
block/qcow2.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 56 insertions(+), 2 deletions(-)
diff --git a/block/qcow2.c b/block/qcow2.c
index d50901675699ef3806bacb780ee0..c8115e1cba0f8ed09254f7889d38 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -5279,6 +5279,38 @@ static int qcow2_load_vmstate(BlockDriverState *bs, QEMUIOVector *qiov,
return bs->drv->bdrv_co_preadv_part(bs, offset, qiov->size, qiov, 0, 0);
}
+static int qcow2_has_compressed_clusters(BlockDriverState *bs)
+{
+ int64_t offset = 0;
+ int64_t bytes = bdrv_getlength(bs);
+
+ if (bytes < 0) {
+ return bytes;
+ }
+
+ while (bytes != 0) {
+ int ret;
+ QCow2SubclusterType type;
+ unsigned int cur_bytes = MIN(INT_MAX, bytes);
+ uint64_t host_offset;
+
+ ret = qcow2_get_host_offset(bs, offset, &cur_bytes, &host_offset,
+ &type);
+ if (ret < 0) {
+ return ret;
+ }
+
+ if (type == QCOW2_SUBCLUSTER_COMPRESSED) {
+ return 1;
+ }
+
+ offset += cur_bytes;
+ bytes -= cur_bytes;
+ }
+
+ return 0;
+}
+
/*
* Downgrades an image's version. To achieve this, any incompatible features
* have to be removed.
@@ -5336,9 +5368,10 @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version,
* the first place; if that happens nonetheless, returning -ENOTSUP is the
* best thing to do anyway */
- if (s->incompatible_features) {
+ if (s->incompatible_features & ~QCOW2_INCOMPAT_COMPRESSION) {
error_setg(errp, "Cannot downgrade an image with incompatible features "
- "%#" PRIx64 " set", s->incompatible_features);
+ "0x%" PRIx64 " set",
+ s->incompatible_features & ~QCOW2_INCOMPAT_COMPRESSION);
return -ENOTSUP;
}
@@ -5356,6 +5389,27 @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version,
return ret;
}
+ if (s->incompatible_features & QCOW2_INCOMPAT_COMPRESSION) {
+ ret = qcow2_has_compressed_clusters(bs);
+ if (ret < 0) {
+ error_setg(errp, "Failed to check block status");
+ return -EINVAL;
+ }
+ if (ret) {
+ error_setg(errp, "Cannot downgrade an image with zstd compression "
+ "type and existing compressed clusters");
+ return -ENOTSUP;
+ }
+ /*
+ * No compressed clusters for now, so just chose default zlib
+ * compression.
+ */
+ s->incompatible_features &= ~QCOW2_INCOMPAT_COMPRESSION;
+ s->compression_type = QCOW2_COMPRESSION_TYPE_ZLIB;
+ }
+
+ assert(s->incompatible_features == 0);
+
s->qcow_version = target_version;
ret = qcow2_update_header(bs);
if (ret < 0) {

3
qemu-6.2.0.tar.xz
View File

@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:68e15d8e45ac56326e0b9a4afa8b49a3dfe8aba3488221d098c84698bca65b45
size 115667324

BIN
qemu-6.2.0.tar.xz.sig
View File

Binary file not shown.

3
qemu-7.0.0.tar.xz Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f6b375c7951f728402798b0baabb2d86478ca53d44cedbefabbe1c46bf46f839
size 125117636

BIN
qemu-7.0.0.tar.xz.sig Normal file
View File

Binary file not shown.

2
qemu-binfmt-conf-Modify-default-path.patch
View File

@ -13,7 +13,7 @@ Signed-off-by: Andreas Färber <afaerber@suse.de>
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/qemu-binfmt-conf.sh b/scripts/qemu-binfmt-conf.sh
index 7de996d536eaf9c41255ae9695a5..cb06245a834f9e8f2bb0464a25ce 100755
index 9cb723f44355c55d21eeeb2c1b7b..02eedc7c92d788637385ef3eba19 100755
--- a/scripts/qemu-binfmt-conf.sh
+++ b/scripts/qemu-binfmt-conf.sh
@@ -334,7 +334,7 @@ BINFMT_SET=qemu_register_interpreter

Some files were not shown because too many files have changed in this diff Show More