pc,pci,virtio: lots of new features
Lots of last minute stuff.
vhost-user-i2c.
vhost-vsock SOCK_SEQPACKET support.
IOMMU bypass.
ACPI based pci hotplug.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
# gpg: Signature made Fri 16 Jul 2021 16:11:27 BST
# gpg: using RSA key 5D09FD0871C8F85B94CA8A0D281F0DB8D28D5469
# gpg: issuer "mst@redhat.com"
# gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>" [full]
# gpg: aka "Michael S. Tsirkin <mst@redhat.com>" [full]
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17 0970 C350 3912 AFBE 8E67
# Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA 8A0D 281F 0DB8 D28D 5469
* remotes/mst/tags/for_upstream3:
vhost-vsock: SOCK_SEQPACKET feature bit support
docs: Add documentation for iommu bypass
hw/i386/acpi-build: Add IVRS support to bypass iommu
hw/i386/acpi-build: Add DMAR support to bypass iommu
hw/arm/virt-acpi-build: Add IORT support to bypass SMMUv3
hw/pci: Add pci_bus_range() to get PCI bus number range
hw/i386: Add a default_bus_bypass_iommu pc machine option
hw/arm/virt: Add default_bus_bypass_iommu machine option
hw/pxb: Add a bypass iommu property
hw/pci/pci_host: Allow PCI host to bypass iommu
docs: Add '-device intel-iommu' entry
hw/virtio: add vhost-user-i2c-pci boilerplate
hw/virtio: add boilerplate for vhost-user-i2c device
bios-tables-test: Update golden binaries
hw/acpi/ich9: Set ACPI PCI hot-plug as default on Q35
bios-tables-test: Allow changes in DSDT ACPI tables
hw/pci/pcie: Do not set HPC flag if acpihp is used
hw/acpi/ich9: Enable ACPI PCI hot-plug
hw/i386/acpi-build: Add ACPI PCI hot-plug methods to Q35
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Merge crypto updates and misc fixes
* Introduce a GNUTLS backend for crypto algorithms
* Change crypto library preference gnutls > gcrypt > nettle > built-in
* Remove built-in DES impl
* Remove XTS mode from built-in AES impl
* Fix seccomp rules to allow resource info getters
* Fix migration performance test
* Use GDateTime in io/ and net/rocker/ code
* Improve docs for -smp
# gpg: Signature made Wed 14 Jul 2021 15:08:00 BST
# gpg: using RSA key DAF3A6FDB26B62912D0E8E3FBE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>" [full]
# gpg: aka "Daniel P. Berrange <berrange@redhat.com>" [full]
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E 8E3F BE86 EBB4 1510 4FDF
* remotes/berrange-gitlab/tags/crypto-and-more-pull-request: (26 commits)
qemu-options: rewrite help for -smp options
qemu-options: tweak to show that CPU count is optional
qemu-options: re-arrange CPU topology options
docs: fix typo s/Intel/AMD/ in CPU model notes
tests/migration: fix unix socket migration
seccomp: don't block getters for resource control syscalls
io: use GDateTime for formatting timestamp for websock headers
net/rocker: use GDateTime for formatting timestamp in debug messages
crypto: prefer gnutls as the crypto backend if new enough
crypto: add gnutls pbkdf provider
crypto: add gnutls hmac provider
crypto: add gnutls hash provider
crypto: add gnutls cipher provider
crypto: introduce build system for gnutls crypto backend
crypto: flip priority of backends to prefer gcrypt
crypto: replace 'des-rfb' cipher with 'des'
crypto: delete built-in XTS cipher mode support
crypto: delete built-in DES implementation
crypto: add crypto tests for single block DES-ECB and DES-CBC
crypto: drop custom XTS support in gcrypt driver
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Testing and plugin updates:
- custom runner playbooks for configuring GitLab runners
- integrate Cirrus jobs into GitLab via cirrus-run
- clean-up docker package lists
- bump NetBSD to 9.2
- bump OpenBSD to 6.9
- make test-mmap more hexagon friendly
- fixup handling of hostaddr for plugins
- disallow some incompatible plugin configurations
- fix handling of -ldl for BSDs
- remove some old unused symbols from the plugin symbol map
- enable plugins by default for most TCG builds
- honour main build -Wall settings for plugins
- new execlog plugin
- new cache modelling plugin
- fix io_uring build regression
- disable modular TCG on Darwin
# gpg: Signature made Wed 14 Jul 2021 15:56:27 BST
# gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44
* remotes/stsquad/tags/pull-testing-and-plugins-140721-5: (44 commits)
MAINTAINERS: Added myself as a reviewer for TCG Plugins
docs/devel: Added cache plugin to the plugins docs
plugins/cache: Added FIFO and LRU eviction policies
plugins/cache: Enable cache parameterization
plugins: Added a new cache modelling plugin
docs/devel: tcg-plugins: add execlog plugin description
contrib/plugins: add execlog to log instruction execution and memory access
contrib/plugins: enable -Wall for building plugins
tcg/plugins: enable by default for most TCG builds
configure: stop user enabling plugins on Windows for now
configure: add an explicit static and plugins check
configure: don't allow plugins to be enabled for a non-TCG build
tcg/plugins: remove some stale entries from the symbol list
meson.build: relax the libdl test to one for the function dlopen
meson.build: move TCG plugin summary output
plugins: fix-up handling of internal hostaddr for 32 bit
tests/tcg: make test-mmap a little less aggressive
tests/vm: update openbsd to release 6.9
tests/vm: update NetBSD to 9.2
tests/docker: expand opensuse-leap package list
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
New test case enum-dict-no-name.json crashes:
$ python3 scripts/qapi-gen.py tests/qapi-schema/enum-dict-no-name.json
Traceback (most recent call last):
[...]
File "/work/armbru/qemu/scripts/qapi/expr.py", line 458, in check_enum
member_name = member['name']
KeyError: 'name'
Root cause: we try to retrieve member 'name' before we check for
missing members. With that fixed, we get the expected error "'data'
member misses key 'name'".
Fixes: 0825f62c84
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210616072121.626431-1-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
The check_aligned_anonymous_unfixed_mmaps and
check_aligned_anonymous_unfixed_colliding_mmaps do a lot of mmap's and
copying of data. This is especially unfriendly to targets like hexagon
which have quite large pages and need to do sanity checks on each
memory access.
While we are at it clean-up the white space and style issues from the
legacy code. As we no longer do quite so much needless memory access
we can also remove the hexagon timeout hack.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Taylor Simpson <tsimpson@quicinc.com>
Message-Id: <20210709143005.1554-27-alex.bennee@linaro.org>
librados-dev is not required by QEMU directly, only librbd-dev.
glusterfs-common is not directly needed by QEMU.
QEMU uses ncursesw only on non-Windows hosts.
The clang package is clang 10.
flex and bison are not required by QEMU.
Standardize on nmap ncat implementation to match Fedora/CentOS.
Remove vim since it is not a build pre-requisite and no other containers
include it.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210623142245.307776-10-berrange@redhat.com>
Message-Id: <20210709143005.1554-18-alex.bennee@linaro.org>
Since Docker Hub has started to enforce pull rate limits on clients, it
is preferrable to use project specific container registries where they
are available. Fedora and OpenSUSE projects provide registries.
The images in these registries are also refreshed on a more regular
basis than the ones in docker hub, so the package update should
generally be faster.
While CentOS also has a registry it is considerably outdated compared
to docker.io, and also only provides x86 images, while docker.io images
are multi-arch.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210623142245.307776-4-berrange@redhat.com>
Message-Id: <20210709143005.1554-12-alex.bennee@linaro.org>
The test aborts and error message as the following be throwed:
"No such file or directory: '/var/tmp/qemu-migrate-{pid}.migrate",
when the unix socket migration test nearly done. The reason is
qemu removes the unix socket file after migration before
guestperf.py script do it. So pre-check if the socket file exists
when removing it to prevent the guestperf program from aborting.
See also commit f9cc00346d ("tests/migration: fix unix socket batch
migration").
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Hyman <huangy81@chinatelecom.cn>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Currently the crypto layer exposes support for a 'des-rfb'
algorithm which is just normal single-DES, with the bits
in each key byte reversed. This special key munging is
required by the RFB protocol password authentication
mechanism.
Since the crypto layer is generic shared code, it makes
more sense to do the key byte munging in the VNC server
code, and expose normal single-DES support.
Replacing cipher 'des-rfb' by 'des' looks like an incompatible
interface change, but it doesn't matter. While the QMP schema
allows any QCryptoCipherAlgorithm for the 'cipher-alg' field
in QCryptoBlockCreateOptionsLUKS, the code restricts what can
be used at runtime. Thus the only effect is a change in error
message.
Original behaviour:
$ qemu-img create -f luks --object secret,id=sec0,data=123 -o cipher-alg=des-rfb,key-secret=sec0 demo.luks 1G
Formatting 'demo.luks', fmt=luks size=1073741824 key-secret=sec0 cipher-alg=des-rfb
qemu-img: demo.luks: Algorithm 'des-rfb' not supported
New behaviour:
$ qemu-img create -f luks --object secret,id=sec0,data=123 -o cipher-alg=des-rfb,key-secret=sec0 demo.luks 1G
Formatting 'demo.luks', fmt=luks size=1073741824 key-secret=sec0 cipher-alg=des-fish
qemu-img: demo.luks: Invalid parameter 'des-rfb'
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The GNUTLS crypto provider doesn't support DES-ECB, only DES-CBC.
We can use the latter to simulate the former, if we encrypt only
1 block (8 bytes) of data at a time, using an all-zeros IV. This
is a very inefficient way to use the QCryptoCipher APIs, but
since the VNC authentication challenge is only 16 bytes, this
is acceptable. No other part of QEMU should be using DES. This
test case demonstrates the equivalence of ECB and CBC for the
single-block case.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Using error_fatal provides better diagnostics when tests
failed, than using asserts, because we see the text of
the error message.
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Since we now require gcrypt >= 1.8.0, there is no need
to exclude the pbkdf test case.
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The main method checks whether the cipher choice is supported
at runtime, so there is no need for compile time conditions.
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
These tests' setUp do not do anything beyong what their base class do.
And while they do decorate the setUp() we can decorate the classes
instead, so no functionality is lost here.
This is possible because since Avocado 76.0 we can decorate setUp()
directly.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210415215141.1865467-4-crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
[PMD: added note to commit message about Avocado feature/version]
Signed-off-by: Cleber Rosa <crosa@redhat.com>
The premise behind the original behavior is that it would save people
from downloading Avocado (and other dependencies) if already installed
on the system. To be honest, I think it's extremely rare that the
same versions described as dependencies will be available on most
systems. But, the biggest motivations here are that:
1) Hacking on QEMU in the same system used to develop Avocado leads
to confusion with regards to the exact bits that are being used;
2) Not reusing Python packages from system wide installations gives
extra assurance that the same behavior will be seen from tests run
on different machines;
With regards to downloads, pip already caches the downloaded wheels
and tarballs under ~/.cache/pip, so there should not be more than
one download even if the venv is destroyed and recreated.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210415215141.1865467-3-crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Some test cases on x86_cpu_model_versions.py are corner cases because they
need to pass extra options to the -cpu argument. Once the avocado_qemu
framework will set -cpu automatically, the value should be reset. This changed
those tests so to call set_vm_arg() to overwrite the -cpu value.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210430133414.39905-8-wainersm@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
This introduces a new feature to the functional tests: automatic setting of
the '-cpu VALUE' option to the created vm if the test is tagged with
'cpu:VALUE'. The 'cpu' property is made available to the test object as well.
For example, for a simple test as:
def test(self):
"""
🥑 tags=cpu:host
"""
self.assertEqual(self.cpu, "host")
self.vm.launch()
The resulting QEMU evocation will be like:
qemu-system-x86_64 -display none -vga none \
-chardev socket,id=mon,path=/var/tmp/avo_qemu_sock_pdgzbgd_/qemu-1135557-monitor.sock \
-mon chardev=mon,mode=control -cpu host
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210430133414.39905-2-wainersm@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Avocado allows us to select set of tests using tags.
When wanting to run all tests using a NetBSD guest OS,
it is convenient to have them tagged, add the 'os:netbsd'
tag.
It allows one to run the NetBSD tests with:
$ avocado --show=app,console run -t os:netbsd tests/acceptance/
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210623180021.898286-4-f4bug@amsat.org>
Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
[PMD: ammend the commit message with example command]
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Add new tests checking the good behavior of the SMMUv3 protecting
2 virtio pci devices (block and net). We check the guest boots and
we are able to install a package. Different guest configs are tested:
standard, passthrough an strict=0. This is tested with both fedora 31 and
33. The former uses a 5.3 kernel without range invalidation whereas the
latter uses a 5.8 kernel that features range invalidation.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210706131729.30749-4-eric.auger@redhat.com>
[CR: split long lines]
[CR: added MAINTAINERS entry]
Signed-off-by: Cleber Rosa <crosa@redhat.com>
When running LinuxTests we may need to run the guest with
custom params. It is practical to store the pxeboot URL
and the default kernel params so that the
tests just need to fetch those and augment the kernel params.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210706131729.30749-3-eric.auger@redhat.com>
[CR: split long lines]
Signed-off-by: Cleber Rosa <crosa@redhat.com>
As the KNOWN_DISTROS grows, more loosely methods will be created in
the avocado_qemu/__init__.py file.
Let's refactor the code so that KNOWN_DISTROS and related methods are
packaged in a class
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Eric Auger <eric.auger@redhat.com>
Message-Id: <20210706131729.30749-2-eric.auger@redhat.com>
[CR: moved aarch64 definition from patch 2 to 1]
[CR: protect get() when arch is not defined]
[CR: split long lines]
Acked-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
If a guest sends binary data on the serial console, we get:
File "tests/acceptance/avocado_qemu/__init__.py", line 92,
in _console_interaction msg = console.readline().strip()
File "/usr/lib64/python3.8/codecs.py", line 322,
in decode (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa9 in position 2: invalid start byte
Since we use the console with readline(), fix it the easiest
way possible: ignore binary data (all current tests compare
text string anyway).
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210515134555.307404-2-f4bug@amsat.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
The tests based on the LinuxTest class give the test writer a ready to
use guest operating system, currently pinned to Fedora 31.
With this change, it's now possible to choose different distros and
versions, similar to how other tags and parameter can be set for the
target arch, accelerator, etc.
One of the reasons for this work, is that some development features
depend on updates on the guest side. For instance the tests on
virtiofs_submounts.py, require newer kernels, and may benefit from
running, say on Fedora 34, without the need for a custom kernel.
Please notice that the pre-caching of the Fedora 31 images done during
the early stages of `make check-acceptance` (before the tests are
actually executed) are not expanded here to cover every new image
added. But, the tests will download other needed images (and cache
them) during the first execution.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210414221457.1653745-4-crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
