kiwi called systemd-cryptsetup directly which does not take
the settings available in /etc/crypttab into account. This
commit changes the activation procedure in a way that the
generator created unit file systemd-cryptsetup@... is used
This Fixes bsc#1219009
- Allow terminal emulation setup from the cmdline
Using rd.kiwi.term will export the TERM variable into the initrd
environment. In case the default value for the terminal emulation
is not appropriate rd.kiwi.term can be used to overwrite the default.
The environment is also passed to the systemd unit which calls
dialog based programs in kiwi dracut code, such that the TERM
setting will be effective there too. For example:
rd.kiwi.term=vt100
This is related to bsc#1218095
- Followup fix for .profile.extra
Allow to source .profile.extra such that it is possible to
read and act on e.g cmdline parameters. This is related to
bsc#1218095
- Add support for reading .profile.extra in initrd
If there is the file /.profile.extra available in the
initrd, kiwi will import this additional environment file
after the import of the standard /.profile file.
This is related to bsc#1218095
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=236
If no console setting is done in the image description for grub
the default output console is set to: gfxterm and the default
input console is set to: console. This Fixes bsc#1219074
- Follow up fix for drop of hybrid boot snippets
The following is left over code from the drop of the hybrid
boot templates.
- Drop hybrid boot snippets from the GRUB 2 configuration template
Sometime between GRUB 2.04 and GRUB 2.06, it became no longer
necessary to use "linuxefi"+"initrdefi" for UEFI boot. The
standard "linux"+"initrd" stanzas work for both legacy BIOS boot
and modern UEFI boot.
Some distributions no longer support "linuxefi"+"initrdefi" at all
anymore, so let's just use "linux"+"initrd" for everything now.
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=235
The commit used a wrong e-mail address which should not
land in the created changes file for the packaging
- Apply changelog fixes by glob
Does not require Makefile changes when maintaining branches
- Add changelog fix file for commit deb6ca
The commit used a wrong e-mail address which should not
land in the created changes file for the packaging
Signed-off-by: David Cassany <dcassany@suse.com>
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=234
The following github actions will be disabled from master
because they are expected to run from main:
- ci-publish-pages.yml
- ci-publish-to-pypi.yml
- ci-update-build-tests.yml
- Bump version: 9.25.21 → 9.25.22
- Fixed regression in GRUB_SERIAL_COMMAND setup
The condition to write the serial line setup was broken.
This commit fixes it. Related to Issue #2419
- Fixed grub terminal setup
The grub terminal setup is divided into the setting for the output
and the input console. For both settings different parameters exists.
So far kiwi did not differentiate between the two parts of the
console setup and that could lead to a wrong setting if only one
value is provided in kiwi's console= attribute which lead to the
grub setting, GRUB_TERMINAL=value. If value is set to e.g gfxterm
grub takes this for both input and output and it's obviously
wrong for the input. To make this less error prune the kiwi code
changes with this commit to set GRUB_TERMINAL_INPUT and
GRUB_TERMINAL_OUTPUT rather than GRUB_TERMINAL and also runs sanity
checks on the provided values if they are applicable. The information
for setting up the console in the schema stays untouched though.
That's because it's used for all bootloaders and also because grub
supports multiple values for the console in/out setting in one
GRUB_TERMINAL variable even though kiwi does no longer use it.
To make this clear for the users also the documentation for the
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=232
Incorporate latest changes from upstream MicroOS and
also add an encrypted profile build for testing
- Bump version: 9.25.18 → 9.25.19
- Fix typo in workflow overview doc section
- Add missing documentation for oem-unattended-id
- Allow install disk overwrite from cmdline
Add rd.kiwi.oem.installdevice=DEVICE. Configures the disk device
that should be used in an OEM installation. This overwrites any
other oem device setting, e.g device filter or maxdisk and just
continues the installation on the given device. However, the
device must exist and must be a block special.
This Fixes jira#PED-7180
- Update mailmap
- Replace the regex-based loader entry fix with string parsing (#2388)
A user building RHEL images ran into issues with the initrd.
It turns out that RHEL uses some patches that mean the
initrd/linux files in RHEL are not installed to /boot, which trips
up the original regex. The new fix doesn't rely on matching the
path in boot, instead just finding the initrd/linux files and rewriting
them in place.
This change also adds the pre-and-post fix loader entries to the debug logs.
Reference: https://bugzilla.suse.com/1208701
Fixes suse bsc#1208701
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=231
- Fixed argument processing of config.bootoptions
Instead of adding an extra space to make the subsequent reading
to work, use an awk script that does it without nasty workarounds
- Allow test-image-disk-ramdisk to be build locally
- Bump version: 9.25.15 → 9.25.16
- Fixed doc search field
The search in the documentation is broken with the rtd
schema and latest sphinx. Stick with sphinx 5.0.0 for the
moment which fixes the issue
- Bump version: 9.25.14 → 9.25.15
- Make codacy happy
- Support release version in signkey URLs
Using one of the $releasever/${releasever} variable placeholders
in an URL as part of a <signing key="..."/> element did not replace
the placeholder with the value of the <release-version> element.
This commit fixes this and also makes sure that the result list
for downloading signing keys is unique. This Fixes#2381
- Bump version: 9.25.13 → 9.25.14
- Update pypi release to trusted workflow
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=230
- Fix double grub entries in hybrid ISO images
As consequence of using the "source" grub command instead
of the "configfile" command to load the grub config we now
see double loading of the same file on ISO media. The reason
here is that kiwi ISO media is always hybrid which means it
embeds an MBR into the ISO for which the "source" command now
can read the same file through two different device paths.
This does not happen with the "configfile" grub command.
Thus this patch uses "configfile" if we produce an ISO image
and "source" for all other image types. The commit also fixes
the custom grub template used for ISO images in a way that
we only set the "serial" command if there is a serial
configuration provided along with the image description.
- Update x86/tumbleweed/test-image-systemd-boot
systemd package has been splitted and provides
systemd-boot in an extra package now
- Limit scope of grub _fix method
Only apply _fix_grub_to_support_dynamic_efi_and_bios_boot for
the x86_64 or ix86 architectures. This Fixes#2343
- Bump version: 9.25.10 → 9.25.11
- remove trailing whitespace
- rpm spec: add missing requires of typing_extensions for python < 3.8
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=229
- dracut: modules.d: 99kiwi-lib: add bash shebangs and dependency
The scripts in the kiwi-lib module use bash-specific syntax like
`function`, which causes the script to fail if another shell (like dash
or busybox) is used to interpret the scripts. Specifically set the shebang
to use bash as the shell interpreter and add bash as a dependency to the
dracut module to fix this.
This resolves the following errors which are printed at boot when
busybox tries to interpret the scripts:
/bin/dracut-pre-mount: 3: /lib/kiwi-filesystem-lib.sh: function: not found
/bin/dracut-pre-mount: 15: /lib/kiwi-filesystem-lib.sh: Syntax error: "}" unexpected
Signed-off-by: Isaac True <isaac.true@canonical.com>
- Do not depend on dnf for SUSE ALP
- Added appx-util as WSL requires for Fedora/RHEL
Make sure the kiwi-systemdeps-containers-wsl meta package
pulls in the required tools for Fedora/RHEL when building
WSL containers. This is a followup to #2286
- Add cpio to core systemdeps requires list
- Bump version: 9.24.59 → 9.24.60
- Allow to create plain cpio archives
Allow to create plain cpio archives to serve the idea
of the mkosi initrd concept
- Fix restoration of grub2-install (bsc#1210948)
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=228
- Adding a comment to explain the s390 specific path
Signed-off-by: David Cassany <dcassany@suse.com>
- Use parted only in s390 arch
This commit ensures parted is only called in dracut-kiwi-lib for s390
arch. It also fixes the spec to only require parted for
s390 builds.
In addition parted requirement has been removed from dracut-kiwi-live
package since parted is not used in any other dracut module beyond
kiwi-lib.
Signed-off-by: David Cassany <dcassany@suse.com>
- Expand DASD partition table with parted
Signed-off-by: David Cassany <dcassany@suse.com>
- Fixed evaluation of custom start sector
In case an alternative partition table start sector is
configured, the check to effectively apply it should be
based on the partition table type not on the firmware
name
- Add parted dependency for s390
Signed-off-by: David Cassany <dcassany@suse.com>
- Bump version: 9.24.57 → 9.24.58
- Build leap btrfs test without root as snapshot
So far no btrfs integration test without root as snapshot existed
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=227
- Allow squashfscompression for plain squashfs
The schematron rule to limit the squashfscompression attribute
to certain image types did not allow it for a plain squashfs
filesystem build. This commit fixes that limitation.
This Fixes#2241
- Use rsync defaults to sync the initrd root-tree
This commits makes use of rsync default options to sync
the root-tree of the boot image for custom initrds.
Fixes bsc#1207128 where it was noted hardlinks
were not preserved inside the initrd.
Signed-off-by: David Cassany <dcassany@suse.com>
- Fixed wrong test assertions
Former versions of pytest did ignore invalid assertions.
Now they are reported as an error and show the mistakes
of the past
- Added 90-brd.conf for test-image-disk-ramdisk
By default the brd ramdisk module is no longer added to the
initrd. For ramdisk deployments this is required though.
This Fixes#2230
- Delete CentOS v7 build tests
CentOS v7 is EOL for kiwi image build tests
- Allow to specify fixed size for the root partition
So far the last partition (typically root) in the partition table
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=226
- Allow to pass credentials as file reference
- Fixed handling of container archive compression
In kiwi we support handling of the container archive compression
via a runtime configuration setting of the following form, eg
in /etc/kiwi.yml
```yaml
container:
# Specify compression for container images
# Possible values are true, false, xz or none.
- compress: true
```
However, this setting was only taken into account in the kiwi
bundler. Meaning if the user calls 'kiwi result bundle ...'
after the container image has been created the result bundler
will take the compression setting into account. From my perspective
this behavior is misleading and also prevents users from
creating compressed container archives without a subsequent
result bundler call. Therefore this commit moves the place to
handle the compression setting into the container classes.
The bundler code for containers will no longer operate on it
and just takes what it gets, which can be either compressed
or not. The default setting was "No compression" and this
was not changed. This Fixes#2217
- Add type hints for OCI and APPX classes
- Bump version: 9.24.53 → 9.24.54
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=225
- Fixed test-image-vagrant
virtualbox-guest-tools obsoletes virtualbox-guest-x11
- ignore the type check on the Result class
With an update of mypy the bound TypeVar is no longer allowed.
In newer versions of python we could use the "Self" type or
import annotations from the future module. Unfortunately in
older python versions which we still support (3.6) there is
no non intrusive change which allows us to handle that type
annotation. Thus this commit ignores the return type spec
for Result.load() for the moment.
- Stop copying /dev files statically into the OCI container
In containers (nspawn) where part of the /dev filesystem is bind-mounted
from outside system, kiwi fails to do the rsync (in creation of the
nodes).
There is no reason to actually copy whole tree inside so let's just
not do it (as it does not seem to be needed at all).
- List riscv64 as a valid architecture in the schemas
This is needed so that architecture filters on riscv64 specifics can
be defined.
- Support DM integrity legacy options
Add a new attribute integrity_legacy_hmac="true|false" which
allows to use old flawed HMAC calculation (does not protect superblock).
Add a new attribute integrity_legacy_padding="true|false" which
allows to use inefficient legacy padding. Do not use these attributes
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=224
- Support URIs with credentials in solver plugin
check if the URI string contains credentials and
extract/trim them from the uri object. The urlparse
class does not recognize this information as a valid
URI and throws an exception
- Fixed test-image-disk-legacy
Increase size of embedded EFI fat image needed for EFI
bootable install ISO. Due to the increased size of the
custom kiwi initrd the default size for the FAT image
is too small
- Support custom size for embedded EFI FAT image
For ISO images (live and install) the EFI boot requires an embedded
FAT image. As of now a fixed size of 20M was used which leads to a
problem if the EFI image or the initrd or the kernel is bigger than
20M. With the new attribute:
efifatimagesize="nonNegativeInteger"
we can now set a different value for the FAT image. Please note the
value must be aligned to the also customizable efipartsize value
which allows to configure the size of the EFI partition
- Fixed linter complaints after flake update
flake now complains about E275 missing whitespace
after keyword for the way 'del' is used. This commit
fixes it
- Added a NOTE: section to explain the finer points of adding tools to a
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=221
- Umount device before cloning
In case a clone should be created from a partition we need
to make sure to umount the device after sync and prior cloning.
Otherwise the clone operation is not safe because the rsynced
data might still be in memory and not synced out to the block
device.
- Fixed custom disk start sector setup
The attribute disk_start_sector allows to specify a custom
start sector for the first partition of the disk. On GPT
tables everything works nicely, on DOS tables the used tools
fdisk/sfdisk are not able to manage the start/end values of
subsequent partitions if the first partition doesn't start
with the tooling default. This patch allows to set the
start sector after the partition table has been created
- Fixed CentOS-8 repo setup
- Bump version: 9.24.43 → 9.24.44
- Make sure to rebuild rpm database
For rpm based distributions make sure to call
rpm --rebuilddb unconditionally prior using rpm
with the chroot. This Fixes#2165
- github: Refresh issue template to cover more operating scenarios
KIWI is often used for cross-distribution image builds, so we
should ask for that information when appropriate.
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=220
- Fix helper method to detect dracut outfile format
The method _get_boot_image_output_file_format_from_dracut_code
is used in kiwi to match parts of the dracut code for the used
output file format. Beginning with dracut-056 the code part
checked has changed syntactically such that the match did
no longer work. This commit increases the scope of the match
and replace pattern and Fixes#2149
- Fixed handling of signing_keys in cmdline options
When passing signing_keys with the --add-repo|--set-repo
commandline options the delimiter to separate the single
key information is a colon(:). However, this is stupid when
kiwi expects the signing key to be references as an URI
format like file://... Therefore this patch changes the
delimiter from colon(:) to semicolon(;)
- Bump version: 9.24.39 → 9.24.40
- Setup SELinux on every system prepare / build (#2148)
Setup SELinux on every system prepare / build such that all image types benefit from it not only the disk (oem) type
- Install all of QEMU to Ubuntu arm integration test
- rename user to ubuntu for Ubuntu integration test
- Bump version: 9.24.38 → 9.24.39
- Move to sphinx>=5.0.0
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=219
- Subformats should also not be compressed when encryption is enabled (#2138)
Subformats should also not be compressed when encryption is enabled
This is a follow on change to bdba953. When the filesystem is encrypted the
resulting image should not be compressed. Also explain why we ignore the
compression seeting in the user configuration for encrypted images.
- Add support for prebuilt bootstrap package for apt
When using the apt packagemanager kiwi required the use of
debootstrap to create the initial rootfs. This works as long
as there is always a main distribution repository available
which follows the structure of the official debian mirrors.
However if such a main distribution is not present or an
alternative layout like e.g OBS repos is used, debootstrap
will refuse to work. To allow for an alternative and without
the dependency to debootstrap kiwi supports using a prebuilt
bootstrap package providing the mini rootfs to serve as
the bootstrap result. As all other package managers properly
supports installation into an empty new root, this feature
was only added when using the apt packagemanager
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=218
- Allow more repo params to be set on the cmdline
The repository parameters for signing keys, the component
list the main distribution name for debian repositories and
also the repository_gpgcheck could not be set via the
commandline options --add-repo and/or --set-repo. This
commit adds support for them and also updates the manual
page accordingly
- Update ubuntu integration tests
Build them against latest release (jammy).
This Fixes#2128
- Add support for partition cloning
Support creating block level clones of certain partitions
used in the image. Clones can be created from the root, boot
and any partition listed in the <partitions> element.
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=217
- Follow up fix for isolinux-config
isolinux-config is called to update the search path inside
of the isolinux binary. isolinux/syslinux is exclusive to
the ix86 architecture and to BIOS firmware. Therefore the
condition to actually call it should reflect this.
- Bump version: 9.24.32 → 9.24.33
- Fixed runtime check
Fixed check_dracut_module_for_disk_overlay_in_package_list. The
check complains if the dracut-kiwi-overlay module is not installed
but overlay support was requested. This is correct but should only
be done if the selected initrd system is dracut.
- Add option to set LUKS type to luks1 (#2126)
Add option to set LUKS type to luks1
So far the LUKS type could be set to luks and luks2. However, what luks
version the value 'luks' evaluates to depends on how the distributor has
packaged luks. Thus it's possible that 'luks' is either luks1 or luks2. To
also have the opportunity to explicitly specify luks1 this commit adds
the opportunity in the schema.
- Update devel packages helper
Added trang as needed when working on the schema
- Add support for dm integrity with secret key
Allow to protect the opening of the integrity data map and
journal through a keyfile. For setting the key file two new
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=216
- Fix LABEL detection (#2112)
When only "root=" is specified on the kernel command line the match is
located in the first group. Loop through the groups upon mtach to find
what we are looking for.
- Preserve LABEL setting (#2108)
Preserve the LABEL= setting when the grub config file is re-generated.
the GRUB_ENABLE_LINUX_LABEL setting does not exists upstream and
not in any SUSE distribution. Set the grub setting such that LABEL
is preserved on SUSE distros. (bsc#1197616)
- Fix test_setup_default_grub_empty_kernelcmdline
The unit test exists to check that GRUB_CMDLINE_LINUX_DEFAULT
is not set depending on the provided cmdline. The test exists
for reasons explained in Issue #1650
- Don't compress .appx containers (#2106)
The container is actually inside and already compressed.
- Added new CloneDevice class
Added CloneDevice class to the storage interface.
The class allows to create clone(s) from a given source
block device into a list of target block devices.
The target block devices are clones of the source but
prevents device naming conflicts for unique identifiers
like the UUID. This is requires to still allow to boot
from images containing device clones and needs to be
handled by tools that might work on top of the cloned
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=215
- Don't bind mount /run during build time
In commit #9512318 a new bind mount of /run into the root tree
during build time was introduced. The bind mount was done because
in my tests running podman from config.sh it did not work without
/run bind mounted. However, it turned out that I was wrong because
along with the provided methods to prepare cgroups and a custom
runtime configuration method; setupContainerRuntime() it is not
needed to have /run bind mounted. Thus this commit deletes the
bind mount of /run and therefore Fixes#2067
- Fix github action running obs service refresh
The curl command to send the POST request for running the
obs remote service uses the --fail-with-body option.
Unfortunately the ubuntu-latest container used to run the
action comes with a curl version that does not support the
option. Thus this commit removes the use of the option
- Style changes in container docs
Reformulate the container building guide a bit
- Update schema docs
Signed-off-by: David Cassany <dcassany@suse.com>
- Provide schema version v7.5 in spec
- Update descriptions to schema v7.5
- Update cron for security scorecard
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=214
- Fixed name of secret variable
The ci-update-build-tests action used a wrong variable
name which does not exist in the github secrets. This
commit fixes it and uses the correct variable name
- Bump version: 9.24.21 → 9.24.22
- Revert "Revert "Fixed MicroOS build test""
This reverts commit 8c4464b8ff2af2642439ce92e1e2be497f2b0f4d.
snapper default config has moved from /etc to /usr/share
now hopefully for the last time
- Fixed unit tests
The pytest interface setup() method call has changed
in a way that an additional parameter is passed to
the method which leads to a python error at invocation
time if the setup method does not define it.
- build-tests: Update CentOS 8 test appliance to CentOS Stream 8
CentOS Linux 8 is now EOL, so switch over to CentOS Stream 8.
- Fixed handling of oem reboot settings
There are oem settings called oem-reboot, oem-reboot-interactive
as well as oem-shutdown and oem-shutdown-interactive. When used
the information is passed along to the profile but not evaluated
by any initrd code. I don't know where on the way we lost the
code that actually works with these settings but this commit
makes them effective. This Fixes#2056
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=213
- Fixed UUID used in grub early boot script
In case the system is luks encrypted the UUID of the root
partition was used in the grub early boot script. However,
this condition is only correct if in addition to the luks
encryption the boot area is on crypto too. If boot is not
on crypto the UUID must be the boot partition and not root.
Only if root AND boot is on crypto the kiwi created early
boot script includes the grub cryptomount calls.
- Bump version: 9.24.19 → 9.24.20
- Followup fix for force deleting debian packages
The force uninstall deletes pre scripts prior removal
because if they fail the package will not be removed.
For a force uninstall we consider this ok. However,
the deletion of the scripts did not happen in the
image root. This patch fixes it
- Followup fix for force deleting debian packages
Pass --force-depends to allow uninstall even if the
dependency checker complains
- Fix use of xattrs for container sync
when syncing data for containers only a subset of xattr
attributes can be applied. This Fixes#2009
- Bump version: 9.24.18 → 9.24.19
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=212
- Disable intersphinx
intersphinx is a doc extension which links to the documentation of
objects in other projects whenever Sphinx encounters a cross-reference
that has no matching target in the current documentation set, it
looks for targets in the documentation sets configured in the
intersphinx_mapping. However, the kiwi docs do not use this feature
thus it can be disabled.
- Compress container images in builder class
This commit changes the stage at which container images are compressed
to match the criteria applied to other image types. Instead of
compressing the image in OCI devoted classes now it is happening
in builder class by setting Result instance properties.
Fixes#1996
Signed-off-by: David Cassany <dcassany@suse.com>
- Revert "Fixed MicroOS build test"
This reverts commit 52c38f9ec22aef484efaf0a570dc78eea529deec.
The TW people moved to an older version of libsnapper, no
idea what these guys are doing
- kiwi-repart: Fix spelling error in source code comment
- Fix multiple minor spelling errors in documentation
- Added support for setting up release version
Currently the release version is not set or set to '0'
for package managers which requires a value to operate.
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=210
- Added documentation about sharing backends
The kiwi boxbuild plugin supports several sharing
backends. Details about them and eventual pre-setup
steps prior use are mentioned in this document
- Fixed MicroOS build test
snapper default config has moved from /etc to /usr/share
- Bump version: 9.24.8 → 9.24.9
- Bump version: 9.24.7 → 9.24.8
- Added support for reading metalink in info module
For resolver operations through libsolv the 'kiwi image info'
module exists. So far it could not read the repos from
metalink repo definitions. This Fixes#1890
- Force key attribute from signing element to be treated as a URI
Signed-off-by: David Cassany <dcassany@suse.com>
- Pass signing keys from the XML to the repositories
This commits makes sure signing keys are passed to
repositories setup in build task.
Signed-off-by: David Cassany <dcassany@suse.com>
- Move static sle15 integration test to git
- Bump version: 9.24.6 → 9.24.7
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=209
- Followup fix for debootstrap called only once
A recent change skipped calling debootstrap if the allow-existing-root
flag was passed in combination with apt as the package manager.
However this is not enough. If you say allow-existing-root but
the existing root is empty or not valid to continue with chroot
and apt the debootstrap phase should not be skipped. This commit
checks if apt works in the chroot such that we can assume
debootstrap has done its job and can be skipped
- Bump version: 9.24.4 → 9.24.5
- Fixed include processing
This commit fixes several issue connected with the use of
the <include> directive:
First and foremost the XSLT chain was broken in a way that
the include XSLT in combination with the PrettyPrinter XSLT
were called not in the chain of stylesheets but together.
This results in XML descriptions which duplicated the content
and went invalid
Another change is, when the include XSLT is called in the chain.
This commit moves it to become the very first processing
instruction such that the included data is part of all subsequent
XSLT stylesheets. This also allows to use older schema versions
in included XML data and they get automatically converted through
the chain of XSLT stylesheets.
Last change is the evaluation of the from= attribute value. This
value is now interpreted as an URI. Currently only local URIs are
supported. The reason to do this is because XSLT when processing
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=208
- Make sure embedded test exposes the kernel
- Fixed use of LUKS encrypted images with empty pass
For initial provisioning of LUKS encrypted disk images an
empty passphrase key is handy to avoid interaction in the
deployment process. However, the dracut kiwi modules were
lacking the information that the luks keyfile could be an
empty passphrase key which must not be opened with the
potential risk to get prompted for input. This commit
introduces a new profile environment variable evaluated
by the dracut kiwi lib code to open the LUKS pool and
allows to distinguish the situation on key files with
or without a passphrase
- Fixed type annotations for LuksDevice class
- Fixed repo setup for test-image-embedded
Use obsrepositories
- Added a build test for no-initrd / no-bootloader
It's allowed to configure an image building without an
initrd and/or bootloader setup. These are settings mostly
used in the embedded world and this build test makes sure
images of that kind can be build
- Add support for portable result data (#1949)
In addition to the serialized Result instance kiwi.result
file this commit also creates a portable version of this
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=207
- Add support for toplevel include directive(s)
On the toplevel of an image description you can now
specify include directive(s) like in the following
example
<image ...>
...
<include from="filename_a.xml"/>
...
<include from="filename_b.xml"/>
</image>
At the place of their occurrence the include statement
will be replaced with the contents of the given filename.
The implementation is based on a XSLT stylesheet and
applies very early in the process. The stylesheet reads
the contents of the given file as XML document().
Thus only valid XML documents gets accepted by this
include concept. This Fixes#1929 and is related to
Issue #1918
- Bump version: 9.23.62 → 9.23.63
- Fixed error reporting for stateful description
Errors due to missing or no type definitions were reported
provding the internal object reference of the XML parse
result. This is useless information for users and needs
to be done better. This commit fixes the error message to
avoid showing object references and includes information
about the applied profiles used for this XML state.
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=206
- Only wipe bundle dir when required
The given result bundle dir must only be wiped if the
request to turn the result files into an rpm was given.
Only in this case the given bundle dir must start empty
- Fixed uninstall handling via dnf, microdnf, zypper
The above package managers supports uninstall instructions
like 'iwl*'. In kiwi there was code checking via rpm if
the packages given to uninstall actually exists. That code
does not work if the given package to uninstall is an
instruction that matches a pattern. Therefore if we use
the uninstall section in the kiwi image description, just
pass the provided information to the package manager and
don't try to be clever in kiwi itself.
- Allow to set --logfile for result namespace
Setting a logfile for e.g 'kiwi-ng result bundle ...'
is useful and should be possible
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=205
- Added support for building rpm package from bundle
With the new option --package-as-rpm it is possible to
call the kiwi result bundler such that the image build
results gets packaged into an rpm. I think this is a
handy feature to transport image builds via repositories
- Fixed MicroOS integration test
With ignition/combustion in place it's not allowed
to use tmp as a subvolume
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=204
- Fixed condition for GRUB_DISABLE_LINUX_UUID="true"
The grub config parameter GRUB_DISABLE_LINUX_UUID must only
be set if the device persistence setting is not UUID. However,
in kiwi UUID device names are the default and doesn't have to
be expressed explicitly. Therefore the condition to check
for different than 'by-uuid' is wrong for the default case were
no device persistence setting exists. This results in a wrong
grub option to be set. This commit fixes it in a way to disable
UUID device names in grub if the only other device persistency
setting in kiwi named: 'by-label' is explicitly configured.
This Fixes#1842
- Added force_trailing_slash argument to sync_data
A speciality of the rsync tool is that it behaves differently
if the given source_dir ends with a '/' or not. If it ends
with a slash the data structure below will be synced to the
target_dir. If it does not end with a slash the source_dir
and its contents are synced to the target_dir. For example:
source
└── some_data
1. $ rsync -a source target
target
└── source
└── some_data
2. $ rsync -a source/ target
target
└── some_data
The parameter force_trailing_slash in the DataSync::sync_data
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=203
- Fixed fstab entry for swap on LVM
If an LVM setup is used together with a swapspace the
swap is created as a volume in the volume group. The
required fstab entry to activate swap was not using
the LVM exposed device but the UUID of the low level
block layer. This low level device is not created
by udev because LVM takes over precedence in this
case.
- Bump version: 9.23.47 → 9.23.48
- Fixed LVM get_volumes
Due to the change on the temporary directory yet another
error in the volume manager classes for LVM got exposed.
- Bump version: 9.23.46 → 9.23.47
- Fixed LVM/btrfs volume based image builds
Due to the change on the temporary directory an error
in the volume manager classes for LVM and btrfs was
exposed. There was code which uses a sub-part of the
volume mount point directory which is a tempdir based
directory name. The sub-part was choosen by an index
based path split which worked for /tmp but no longer for
/var/tmp. With tempdir now being also a commandline
option this code has to become more robust.
- Bump version: 9.23.45 → 9.23.46
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=202
- Fixed fedora integration test builds
Maintain the repos in the obs prj config which prevents
the weird "nothing provides kernel-obs-build" error
- Bump version: 9.23.40 → 9.23.41
- Remove util-linux-systemd & util-linux Requires from dracut-kiwi-overlay
These dependencies are pulled in via dracut-kiwi-lib.
- Add missing util-linux-systemd Requires to dracut-kiwi-[live,libs]
- Fixed test-image-orthos integration test
The test was missing btrfs_root_is_snapshot which is required
when using btrfs on tumbleweed.
- Fixed test-image-disk-legacy integration test
The test did not set a device filter for ramdisk devices but
activates unattended mode. In this mode the first device in
the list is taken and this is a ramdisk device which is
by default too small to be used for the installation. Thus
the install usually fails. This commit sets the device filter
for ramdisk devices such that only associated disk devices
can be used for the install process, which is the purpose
of this test. This is related to Issue OSInside/kiwi-functional-tests#8
- Bump version: 9.23.39 → 9.23.40
- Mount dev and proc filesystems prior dracut
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=200
- Stop plymouth also for progress dialogs
- Prevent explicit man page compression
The manual pages are compressed by the packager tooling.
There is no need to do this ourselves
- Bump version: 9.23.29 → 9.23.30
- Increase integration tests boot timeout
Increase integration tests timeout from 2sec to 10sec
- Update integration tests to be non interactive
Some integration tests allows for interactive dialogs on the
bootloader menu or in the installation process. As we plan to
use these tests for automated functional testing there should
be no interaction whenever possible. This Fixes#1811
- Update kiwi installation documentation
The installation chapter contained information about the manual
install of package keys. That information is suspect to be always
outdated because these keys changes. Instead of describing the
manual install of the package key the docs moved to use the
auto-import feature of the package manager. As the instructions
were also rpm specific but we also support install via other
package mangers the complete chapter was a bit reworked and
should be more straight forward now. This Fixes#1799
- Update documentation qemu calls
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=199
- Upgrade tests accoring to #1805
This commit is a follow up of #1805 which missed to update the related
unit test.
- Make installation media unattended
This commit configures install media of several tests to run unattended
installation. This is done to facilitate the logic of functional tests.
- Fixed unit tests for parallel invocation
With the change to allow the platform architecture to be
set application global, the unit tests might fail if tests
run in parallel and using different mock architectures
for the test. Thus test that runs depending on a platform
name needs to set the name in the test
- Update Debian integration test for UEFI testing
Updated the Virtual disk profile to make use of the EFI
secure boot feature.
- Fix WSL appx filemap relative paths not preserved
During WSL appx image type creation step the file hierarchy under metadata_path
is written to a temporary file for eventual use as argument to utility appx.
The file hierarchy information is dropped resulting in all filemap entries
appearing to be at the metadata_path root. The resulting image will side load
and run but without icon and other resources. Stricter checks at Windows Store
submission will fail due to mismatch between image manifest and contents.
Fix by preserving relative path of filemap entries relative to metadata_path.
Add log output showing both input absolute path and output relative path.
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=198
- Update zypp.conf architecture setting
Make sure the architecture is set as parameter in the
zypp.conf file used for building the image. This is needed
to allow differentiation between host arch and image arch
for cross image build environments
- Added option to set the image target architecture
The option --target-arch allows to set the architecture
used to build the image. By default this is the host
architecture. Please note, if the specified architecture
name does not match the host architecture and is therefore
requesting a cross architecture image build, it's important
to understand that for this process to work a preparatory
step to support the image architecture and binary format
on the building host is required and is not considered a
responsibility of kiwi. There will be a followup effort
on providing a plugin for kiwi which should be used to
manage the needed binfmt settings for cross arch image
builds
- Added openssl to the core requires
openssl is used in kiwi to construct a password hash
if the plaintext password feature for user settings
is used. This Fixes bsc#1184128
- Bump version: 9.23.23 → 9.23.24
- Decommission obsolete code reaching EOL
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=197
- Refactor building of root cmdline setting
Creating the root= cmdline parameter was based on methods
that deals with the uuid. However, it's also possible to
use a label information for the root= cmdline. To support
this kiwi issued a 'blkid --uuid' command but that requires
udev device names to be present on the host. The open
buildservice workers do not run udev and fails apart the
standard. This commit refactors the root cmdline setup
to work with the device node as it exists during build
time such that the blkid call runs against that device
node.
- Bump version: 9.23.20 → 9.23.21
- Delete yum from packagemanager in schema
Auto convert yum to dnf if set as packagemanager. This allows
to delete the yum handling from code parts in kiwi where this
was still present. In addition this fixes the inclusion of yum
into the packagelist. This Fixes#1768
- Add Strong Typing to builder APIs
- This PR add APIs Strong Typing to the repository APIs
Strong Typing has been added to the following files:
repository/base.py
repository/pacman.py
repository/apt.py
repository/dnf.py
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=196
- Fixed API vs. CLI inconsistency
when using kiwi as API the program fails with a usage message
from the Cli class. The kiwi.cli module should not be imported
except for kiwi comandline tasks. It has turned out that the
RuntimeConfig class which is used in several places in different
API classes imports Cli and creates an instance of it to check
for a global option. This causes major issues for all programs
which uses the kiwi API but not the command line interface because
the docopt call in Cli() expects a valid docstring which only
exists in kiwi's cli.py. This commit fixes the inconsistency
and allows people to use the kiwi API independent of any
command line interface. Fixes#1755
- Make sure man pages are part of sdist tarball
The current tarball when uploaded to pypi via gitlab does
not contain the manual pages because the doc target to build
them is not called. This commit adds a doc_man tox target
which is called prior pypi release. This Fixes#1746
- Refactor grub2 installation
This commit refactors grub2 installation method to split it in two
parts. Former grub2.install method was meant to run the grub2-install
tool, however, in addition it was also running the secure boot
installation shim-install. The install method in KIWI is skipped for
those architectures and firmware combinations for which bios support
doesn't exist. This was leading to skip the secure boot installation.
The current approach strips the secure boot installation logic from the
grub2.install method, so skipping the install method does not
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=195
- Fix lsblk flags to get sorted output
This commit modifies the lsblk command flags to get a sorted output
according to the disk layout.
This is related to 176c7eab commita and it fixes bsc#1182264
- Fixed kiwi-systemdeps-filesystems requires
The filesystems requires list also contains low level
tools to manage partitions, loops and subsystems. The tools
to manage LUKS(cryptsetup) are missing and imho belongs there
along with the LVM tools which are listed
- Add strong typing for filesystem classes
This commit strong typing for FileSystem* classes.
Part of #1644
- Added ca-certificates-mozilla to build tests
ca-certificates-mozilla provides the issuer CERT to allow
for https repo connections. The standard ca-certificates is
not enough on suse/leap/sles based integration tests
- Fixed packaging metadata for pypi
Include the README as long description in the metadata
for pypi. The change causes the description on pypi.org
to show the ReST rendered README instead of a message
that the author of the module hasn't provided a description
- Delete legacy oem build test from leap
The legacy custom kiwi boot test will only continue
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=194
- Update documentation
Mention the cross arch build capability of the boxbuild
command. Also update the example box used in the self
container build chapter
- Allow to specify OBS credentials in runtime config
In preparation to the new obs kiwi plugin this commit adds
an opportunity to store obs credentials such that the plugin
could also be used in a non-interactive way
- Update build_status helper for box plugin
The suse box has been renamed to tumbleweed
- Update documentation
Fixed link list to integration tests
- Added rawhide test status to build_status helper
- Added Fedora Rawhide integration tests
- Followup fix for RuntimeConfig refactor
The refactor of the RuntimeConfig made sure the runtime config
file is read in only once. But if the file exists and is empty
after yaml.safe_load like in the kiwi package provided
/etc/kiwi.yml which contains all config options as comments,
the code still reads in the file with every new instance of
RuntimeConfig. This commit fixes this condition
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=193
- Added aaa_base to s390 TW build tests
On s390 TW aaa_base is not pulled in by a dependency in obs.
It looks like the package is pulled in by a file provides which
is not resolved by obs. Thus the package needs to be added
explicitly
- Bump version: 9.23.10 → 9.23.11
- Follow up fix, creating custom grub EFI images
Make sure custom EFI grub image is copied to the media
directory if this is different from the root directory
e.g on creation of live images
- Bump version: 9.23.9 → 9.23.10
- Follow up fix, creating custom grub images
Moving the grub mkimage call as chroot operation also broke
the creation of image builds that uses the legacy custom kiwi
boot image feature instead of dracut. This commit fixes it
- Added leap box to be shown by build_status helper
- Added decorators to help with API management
The lifetime of API methods could be limited due to
the development of kiwi. To allow for a deprecation
process the following helper methods has been added
- Bump version: 9.23.8 → 9.23.9
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=192
- Update contributing link in README
- Bump version: 9.23.3 → 9.23.4
- Fixes the rpmdb bootstrap management for DNF on SUSE
This commit makes sure that the compatibility symlink for
/var/lib/rpm is created when the host rpmdb path is set to something
different. This fixes a mismatch on bootstrapping SUSE using DNF.
Fixes#1669
- Bump version: 9.23.2 → 9.23.3
- Modify lsblk flags for a consistent output across distros
This commit modifies the lsblk flags to make use of the list format
instead of raw output. `--list` flag seams to keep the geometry order
and produces a consistent output in several distros.
- Bump version: 9.23.1 → 9.23.2
- Update kiwi-systemdeps-image-validation
Make python anymarkup to be only recommended. The package
does not exist on all distributions, e.g suse does not
provide it and for kiwi it's an optional plugin
- Fixed missing provides tag
When building WSL images the image type is set to appx.
Therefore obs is looking for what provides kiwi-image:appx
This provides tag was missing
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=191
- Fixed validation of bool value in dracut module
The oem-multipath-scan setup results in a bool variable inside
of the initrd code. The variable kiwi_oemmultipath_scan is
therefore either set to "true" or "false". A check in code
of the form [ -n ... ] is stupid since the variable always
contains text. This commit fixes the validation to make use
of the bool() method provided for these type of variables
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=190
- Omit multipath module by default
The plain installation of the multipath toolkit activates the
dracut multipath code. The setup if the target image runs in a
multipath environment or not should however be decided explicitly
in the image description via <oem-multipath-scan> and not
implicitly by the presence of tools
- Fixed multipath disk device assignment in kiwi lib
The former lookup of the multipath mapped disk device contained
a race condition. If the lookup of the device mapper files happened
before multipathd has finished the initialization, kiwi continues
with the unix node name and fails when the device mapper keeps
a busy state on it. This commit changes the code such that in case
of an explicit request to use multipath the lookup of the mapped
device becomes a mandatory process that runs until the
DEVICE_TIMEOUT is reached. Default timeout is set to 60 sec.
This references Issue SUSE-Enceladus/azure-li-services#255
- Fixed PackageManager decorator in unit test
Implement patch decorators for factories consistently
- Refactor Repository
This commit refactors the Repository class and turns it into a
proper factory class and also includes type hints to facilitate
it's use from an API POV. Related to #1498
- Add DNF as a proper dependency for openSUSE
This is required so that OBS can build openSUSE containers and appliances
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=189
- Fixed dnf plugin config setup
Only create a dnf plugin config if the plugin config directory
to store that file exists in the system
- Set --releasever=0 for microdnf
To allow microdnf to work from an empty root directory
we need to set the release version to zero
- Use custom varsdir for dnf builds
- Partially revert dcounter.c flaw report
I could not find a problem with this read call
it does check on the buffer boundaries and it
only writes the bytes that read returns until
read returns <= 0
- Fixed dcounter.c flaw report
Check buffer boundaries if used in a loop
- Fixed dcounter.c flaw report
Variable scope can be reduced and useless value assignment.
- Fixed microdnf support
The installroot argument must be used together with --config
and additionally with --noplugins, as well as --setopt for
cachedir, reposdir and varsdir. Related to #1625
- Move tools README to ReST
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=188
- Move usrmerge package out of the bootstrap section
Currently bootstrap phase on APT package manager makes use of the
debootstrap tool. However debootstrap is limited to execute the
bootstrap using a single repository. This is causes several limitations
in OBS builds, such as the impossibility of using update repositories or
the inclusion of any package that is not part of the standard OBS
repository.
Usrmerge package is part of the universe repository in OBS which is not
te one used by debootstrap, so it can't be installed on bootstrap phase.
- Bump version: 9.21.21 → 9.21.22
- Fixed package manager api inconsistency
The method post_process_install_requests_bootstrap in the
zypper package manager was missing an argument
- Bump version: 9.21.20 → 9.21.21
- Fixed regexp for grub rootdev substitution
The regular expression to match the grub root device
used a lazy glob match ".*?". This however matches a
too long part depending on the rest of the content.
This commit fixes the expression to be strict on
the allowed characters and makes sure the anchor
characters are not part of the matching character
class. This Fixes#1607
- Fix quick start guide build command
OBS-URL: https://build.opensuse.org/package/show/Virtualization:Appliances:Builder/python-kiwi?expand=0&rev=187
