From 93b53787f6719333d62c519d22fcf57a32d9c81e2eaa3ac530ffb034e544911c Mon Sep 17 00:00:00 2001 From: Egbert Eich Date: Fri, 2 Feb 2024 07:10:30 +0000 Subject: [PATCH] Accepting request 1143459 from home:mslacken:sp fix infinite recursion when computing concretization errors + environment: fix an issue with deconcretization/reconcretization of specs + buildcache: don't error if a patch is missing, when installing from binaries In v0.18, we added better error messages that could tell you what problem happened, but they couldn't tell you why it happened. 0.21 adds condition chaining to the solver, and Spack can now trace back through the conditions that led to an error and build a tree of causes potential causes and where they came from. This creates a container image from the Spack installations on the host system, without the need to run spack install from a Dockerfile or sif file. It also addresses the inconvenience of losing binaries of dependencies when RUN spack install fails inside docker build. Further, the container image layers and build cache tarballs are the same files. This means that spack install and docker pull use the exact same underlying binaries. If you previously used spack install inside of docker build, this feature helps you save storage by a factor two. Increasingly, complex package builds require multiple versions of some build dependencies. For example, Python packages frequently require very specific versions of setuptools, cython, and sometimes different physics packages require different versions of Python to build. The concretizer enforced that every solve was unified, i.e., that there only be one version of every package. The concretizer now supports "duplicate" nodes for build OBS-URL: https://build.opensuse.org/request/show/1143459 OBS-URL: https://build.opensuse.org/package/show/network:cluster/spack?expand=0&rev=91 --- spack.changes | 162 ++++++++++++++++++++++++++------------------------ spack.spec | 2 +- 2 files changed, 85 insertions(+), 79 deletions(-) diff --git a/spack.changes b/spack.changes index 287a6a5..6f7111a 100644 --- a/spack.changes +++ b/spack.changes @@ -8,120 +8,125 @@ Thu Jan 25 14:07:19 UTC 2024 - Christian Goll + spack info: sort variants in --variants-by-name + Spec.format: error on old style format strings + ASP-based solver: - fix infinite recursion when computing concretization errors + fix infinite recursion when computing concretization + errors don't error for type mismatch on preferences don't emit spurious debug output + Improve the error message for deprecated preferences + Fix MSVC preview version breaking clingo build on Windows + Fix multi-word aliases + Add a warning for unconfigured compiler - + environment: fix an issue with deconcretization/reconcretization of specs - + buildcache: don't error if a patch is missing, when installing from binaries + + environment: fix an issue with + deconcretization/reconcretization of specs + + buildcache: don't error if a patch is missing, when + installing from binaries - updated to 0.21.0 * following new features: + Better error messages with condition chaining: - In v0.18, we added better error messages that could tell you what problem - happened, but they couldn't tell you why it happened. 0.21 adds condition - chaining to the solver, and Spack can now trace back through the conditions - that led to an error and build a tree of causes potential causes and where - they came from. + In v0.18, we added better error messages that could tell you + what problem happened, but they couldn't tell you why it + happened. 0.21 adds condition chaining to the solver, and + Spack can now trace back through the conditions that led to + an error and build a tree of causes potential causes and + where they came from. + OCI build caches: You can now use an arbitrary OCI registry as a build cache: - ``` $ spack mirror add my_registry oci://user/image # Dockerhub $ spack mirror add my_registry oci://ghcr.io/haampie/spack-test # GHCR $ spack mirror set --push --oci-username ... --oci-password ... my_registry # set login creds $ spack buildcache push my_registry [specs...] - ``` And you can optionally add a base image to get runnable images: - ``` $ spack buildcache push --base-image leap:15.5 my_registry python Pushed ... as [image]:python-3.11.2-65txfcpqbmpawclvtasuog4yzmxwaoia.spack $ docker run --rm -it [image]:python-3.11.2-65txfcpqbmpawclvtasuog4yzmxwaoia.spack - ``` - This creates a container image from the Spack installations on the host - system, without the need to run spack install from a Dockerfile or sif - file. It also addresses the inconvenience of losing binaries of - dependencies when RUN spack install fails inside docker build. - Further, the container image layers and build cache tarballs are the same - files. This means that spack install and docker pull use the exact same - underlying binaries. If you previously used spack install inside of docker - build, this feature helps you save storage by a factor two. + This creates a container image from the Spack installations + on the host system, without the need to run spack install + from a Dockerfile or sif file. It also addresses the + inconvenience of losing binaries of dependencies when RUN + spack install fails inside docker build. Further, the + container image layers and build cache tarballs are the same + files. This means that spack install and docker pull use the + exact same underlying binaries. If you previously used spack + install inside of docker build, this feature helps you save + storage by a factor two. + Multiple versions of build dependencies: - Increasingly, complex package builds require multiple versions of some - build dependencies. For example, Python packages frequently require very - specific versions of setuptools, cython, and sometimes different physics - packages require different versions of Python to build. The concretizer - enforced that every solve was unified, i.e., that there only be one version - of every package. The concretizer now supports "duplicate" nodes for build - dependencies, but enforces unification through transitive link and run - dependencies. This will allow it to better resolve complex dependency - graphs in ecosystems like Python, and it also gets us very close to - modeling compilers as proper dependencies. + Increasingly, complex package builds require multiple + versions of some build dependencies. For example, Python + packages frequently require very specific versions of + setuptools, cython, and sometimes different physics packages + require different versions of Python to build. The + concretizer enforced that every solve was unified, i.e., + that there only be one version of every package. The + concretizer now supports "duplicate" nodes for build + dependencies, but enforces unification through transitive + link and run dependencies. This will allow it to better + resolve complex dependency graphs in ecosystems like Python, + and it also gets us very close to modeling compilers as + proper dependencies. + Cherry-picking virtual dependencies: - You can now select only a subset of virtual dependencies from a spec that - may provide more. For example, if you want mpich to be your mpi provider, - you can be explicit by writing: + You can now select only a subset of virtual dependencies + from a spec that may provide more. For example, if you want + mpich to be your mpi provider, you can be explicit by + writing: hdf5 ^[virtuals=mpi] mpich - Or, if you want to use, e.g., intel-parallel-studio for blas along with an external - lapack like openblas, you could write: + Or, if you want to use, e.g., intel-parallel-studio for blas + along with an external lapack like openblas, you could + write: strumpack ^[virtuals=mpi] intel-parallel-studio+mkl ^[virtuals=lapack] openblas - + License directive: - Spack packages can now have license metadata, with the new license() directive: - license("Apache-2.0") - Licenses use SPDX identifiers, and you can use SPDX expressions to combine them: - license("Apache-2.0 OR MIT") - Like other directives in Spack, it's conditional, so you can handle complex cases like Spack itself: - license("LGPL-2.1", when="@:0.11") - license("Apache-2.0 OR MIT", when="@0.12:") + spack deconcretize command: - We are getting close to having a spack update command for environments, but - we're not quite there yet. This is the next best thing. spack deconcretize - gives you control over what you want to update in an already concrete - environment. If you have an environment built with, say, meson, and you + We are getting close to having a spack update command for + environments, but we're not quite there yet. This is the + next best thing. spack deconcretize gives you control over + what you want to update in an already concrete environment. + If you have an environment built with, say, meson, and you want to update your meson version, you can run: $spack deconcretize meson - and have everything that depends on meson rebuilt the next time you run - spack concretize. In a future Spack version, we'll handle all of this in a - single command, but for now you can use this to drop bits of your lockfile + and have everything that depends on meson rebuilt the next + time you run spack concretize. In a future Spack version, + we'll handle all of this in a single command, but for now + you can use this to drop bits of your lockfile and resolve your dependencies again. + UI Improvements: - The venerable spack info command was looking shabby compared to the rest of - Spack's UI, so we reworked it to have a bit more flair. spack info now - makes much better use of terminal space and shows variants, their values, - and their descriptions much more clearly. Conditional variants are grouped - separately so you can more easily understand how packages are structured. - spack checksum now allows you to filter versions from your editor, or by - version range. It also notifies you about potential download URL changes. - See + The venerable spack info command was looking shabby compared + to the rest of Spack's UI, so we reworked it to have a bit + more flair. spack info now makes much better use of terminal + space and shows variants, their values, and their + descriptions much more clearly. Conditional variants are + grouped separately so you can more easily understand how + packages are structured. spack checksum now allows you to + filter versions from your editor, or by version range. It + also notifies you about potential download URL changes. + Environments can include definitions: - Spack did not previously support using include: with The definitions - section of an environment, but now it does. You can use this to curate - lists of specs and more easily reuse them across environments. + Spack did not previously support using include: with The + definitions section of an environment, but now it does. You + can use this to curate lists of specs and more easily reuse + them across environments. + Aliases: - You can now add aliases to Spack commands in config.yaml, e.g. this might - enshrine your favorite args to spack find as spack f: - ``` + You can now add aliases to Spack commands in config.yaml, + e.g. this might enshrine your favorite args to spack find as + spack f: config: aliases: f: find -lv - ``` + Improved autoloading of modules: - In this release, you can start using hide_implicits: true instead, which - exposes only explicitly installed packages to the user, while still - autoloading dependencies. On top of that, you can safely use hash_length: - 0, as this config now only applies to the modules exposed to the user -- - you don't have to worry about file name clashes for hidden dependencies. + In this release, you can start using hide_implicits: true + instead, which exposes only explicitly installed packages to + the user, while still autoloading dependencies. On top of + that, you can safely use hash_length: 0, as this config now + only applies to the modules exposed to the user -- you don't + have to worry about file name clashes for hidden + dependencies. Note: for tcl this feature requires Modules 4.7 or higher. * Other new commands and directives: - + spack env activate without arguments now loads a default environment that - you do not have to create - + spack find -H / --hashes: a new shortcut for piping spack find output to - other commands + + spack env activate without arguments now loads a default + environment that you do not have to create + + spack find -H / --hashes: a new shortcut for piping spack + find output to other commands + Add spack checksum --verify, fix --add - + New default_args context manager factors out common args for directives - + spack compiler find --[no]-mixed-toolchain lets you easily mix clang and - gfortran on Linux + + New default_args context manager factors out common args for + directives + + spack compiler find --[no]-mixed-toolchain lets you easily + mix clang and gfortran on Linux * Performance improvements: + spack external find execution is now much faster + spack location -i now much faster on success @@ -129,7 +134,8 @@ Thu Jan 25 14:07:19 UTC 2024 - Christian Goll + ASP-based solver: avoid cycles in clingo using hidden directive + Fix multiple quadratic complexity issues in environments * Other new features of note: - + archspec: update to v0.2.2, support for Sapphire Rapids, Power10, Neoverse V2 + + archspec: update to v0.2.2, support for Sapphire Rapids, + Power10, Neoverse V2 + Propagate variants across nodes that don't have that variant + Implement fish completion + Can now distinguish between source/binary mirror; don't ping diff --git a/spack.spec b/spack.spec index 8f05697..7bb9e0d 100644 --- a/spack.spec +++ b/spack.spec @@ -84,7 +84,6 @@ Requires: git Requires: gpg2 Requires: gzip Requires: libbz2-devel -Requires: lua-lmod Requires: make Requires: patch Requires: polkit @@ -94,6 +93,7 @@ Requires: tar Requires: unzip Requires: xz Recommends: %spack_trigger_recommended_packages %spack_trigger_recommended_compilers +Recommends: lua-lmod Requires: (hwloc if hwloc-devel) Requires: (hwloc-devel if hwloc) %else