pool/spack
SHA256
2
0
Fork 1
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Accepting request 1174817 from home:eeich:branches:network:cluster

Browse Source 
- Give 'site' scope a lower precedence than 'system' scope.
  The 'site wide' config scope was meant to be per Spack
  installation. A single system may have multiple Spack
  installations, so was is meant for overriding the
  'system' wide setting per installation.
  The Spack package is OS-vendor provided. The vendor provides
  pr generates a configuration which a local admin may want to
  override. This can now be done from within the 'system' scope.
  Previously the vendor-supplied configuration was mixed with
  the 'system' scope - local modifications collided with
  vendor autoconfiguration.
- Add a build-dependency package which will cause build tools
  and libraries used frequently by Spack to be installed.
  All these packages are recommended by the main Spack package
  already. This package may be used in environments where the
  installation of recommended packages is disabled by default.

- Update Spack to version 0.22.0
  * New features:
    - Compiler dependencies:
      Spack is in the process of making compilers proper dependencies.
      For this, compiler dependencies are moving from `compilers.yaml`
      to `packages.yaml` to make this consistent with other externals.
      For this, dependency graphs will not show the compiler runtime
      libraries like `gcc-runtime` or `libgfortran`.
      To minimize disruption, an existing `compilers.yaml` file will
      continue to work, however, users are encourage to migrate
      before v0.23.
      + Packages compiled with `%gcc` now depend on a new package
        `gcc-runtime`, which contains a copy of the shared compiler

OBS-URL: https://build.opensuse.org/request/show/1174817
OBS-URL: https://build.opensuse.org/package/show/network:cluster/spack?expand=0&rev=104
This commit is contained in:
Christian Goll 2024-06-11 07:38:05 +00:00 committed by Git OBS Bridge
parent 7b15e8c52b
commit f3e15f0c28
6 changed files with 308 additions and 66 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

46
Move-site-config-scope-before-system-scope.patch Normal file
View File

@ -0,0 +1,46 @@
From: Egbert Eich <eich@suse.com>
Date: Thu May 16 18:34:42 2024 +0200
Subject: Move 'site' config scope before 'system' scope
Patch-mainline: Not yet
Git-commit: 55febdb317328ac0e60cb486eac27fe1371cf120
References:
For spack, the 'site' scope if meant for the Spack installation
which a system may have multiple of. Therefore, it lives in
the Spack installation it is meant for. It should override the
'system-wide' settings from the 'system' scope which is usually
located in /etc/spack.
Our Spack package is meant to be system wide, the site scope
will therefore host vendor provided or generated configs which
a local admin should therefore be able to override.
It must therefore have a lower priority that the system scope.
Signed-off-by: Egbert Eich <eich@suse.com>
Signed-off-by: Egbert Eich <eich@suse.de>
---
lib/spack/spack/config.py | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/lib/spack/spack/config.py b/lib/spack/spack/config.py
index 2a2f180f..3eac8542 100644
--- a/lib/spack/spack/config.py
+++ b/lib/spack/spack/config.py
@@ -832,15 +832,15 @@ def create() -> Configuration:
disable_local_config = "SPACK_DISABLE_LOCAL_CONFIG" in os.environ
+ # Site configuration is per spack instance, for sites or projects
+ # No site-level configs should be checked into spack by default.
+ configuration_paths.append(("site", os.path.join(spack.paths.etc_path)))
+
# System configuration is per machine.
# This is disabled if user asks for no local configuration.
if not disable_local_config:
configuration_paths.append(("system", spack.paths.system_config_path))
- # Site configuration is per spack instance, for sites or projects
- # No site-level configs should be checked into spack by default.
- configuration_paths.append(("site", os.path.join(spack.paths.etc_path)))
-
# Python package's can register configuration scopes via entry_points
configuration_paths.extend(config_paths_from_entry_points())

39
run-find-external.sh.in
View File

@ -14,33 +14,44 @@ addpath() {
find_packages() {
if [ $scope = user ]; then
test -e /etc/spack/packages.yaml && mv /etc/spack/packages.yaml /etc/spack/packages.yaml.old
# In case a packages.yaml is left in the system scope
# from an old installation get it out of the way.
if [ ! -e /usr/etc/spack/packages.yaml ]; then
test -e /etc/spack/packages.yaml -a ! -e /etc/packages.yaml.rpmsave && \
mv /etc/spack/packages.yaml /etc/spack/packages.yaml.rpmsave
rm -f /etc/spack/packages.yaml
fi
test -e /usr/etc/spack/packages.yaml -a ! -e /usr/etc/packages.yaml.rpmsave && \
mv /usr/etc/spack/packages.yaml /usr/etc/spack/packages.yaml.rpmsave
# drop all root rights, when calling external find
sudo -u nobody PATH=${mypath}:${PATH} spack external find --scope user --all #--exclude 'installdbgsymbols'
if [ -e /var/lib/nobody/.spack/packages.yaml ] ; then
mv -v /var/lib/nobody/.spack/packages.yaml /etc/spack/packages.yaml
chown root:root /etc/spack/packages.yaml
chown root:root /var/lib/nobody/.spack/packages.yaml
mv -v /var/lib/nobody/.spack/packages.yaml /usr/etc/spack/packages.yaml
rm -rf /var/lib/nobody/.spack
fi
rm -r /var/lib/nobody/.spack
else
# May run in a container...
PATH=${mypath}:${PATH} spack external find --scope system --all #--exclude 'installdbgsymbols'
fi
test -e /usr/etc/spack/packages.yaml.rpmsave && \
{ diff -q /usr/etc/spack/packages.yaml.rpmsave /usr/etc/spack/packages.yaml &> /dev/null && \
rm /usr/etc/spack/packages.yaml.rpmsave; } || true
}
find_compilers() {
if [ $scope = user ]; then
test -e /etc/spack/compilers.yaml && mv /etc/spack/compilers.yaml /etc/spack/compilers.yaml.old
# drop all root rights, when calling external find
sudo -u nobody PATH=${cpath}:${PATH} spack compiler find --scope user
if [ -e /var/lib/nobody/.spack/compilers.yaml ] ; then
mv -v /var/lib/nobody/.spack/compilers.yaml /etc/spack/compilers.yaml
chown root:root /etc/spack/compilers.yaml
fi
else
# May run in a container...
PATH=${cpath}:${PATH} spack compiler find --scope system
test -e /etc/spack/compilers.yaml \
-a ! -e /etc/spack/compilers.yaml.rpmsave && \
mv /etc/spack/compilers.yaml /etc/spack/compilers.yaml.rpmsave
rm -f /etc/spack/compilers.yaml
fi
diff -q /etc/spack/compilers.yaml.rpmsave /etc/spack/compilers.yaml &> /dev/null \
&& rm /etc/spack/compilers.yaml.rpmsave
test -e /etc/spack/compilers.yaml.rpmsave && \
echo -e "/etc/spack/compilers.yaml.rpmsave exists,\n"\
" compilers are handled in /etc/spack/packages.yaml, now.\n"\
" Please check files for compilers you may have added manually" || true
}
if [ -e /etc/spack/no_rpm_trigger ] ; then

3
spack-0.21.2.tar.gz
View File

@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b1860537ba00c55fa0b2517ce9dbfe0e415600892c48e3dc4e15ee8da0f50dd3
size 10614896

3
spack-0.22.0.tar.gz Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:81d0d25ba022d5a8b2f4852cefced92743fe0cae8c18a54e82bd0ec67ee96cac
size 11101704

207
spack.changes
View File

@ -1,3 +1,210 @@
-------------------------------------------------------------------
Thu May 16 17:03:50 UTC 2024 - Egbert Eich <eich@suse.com>
- Give 'site' scope a lower precedence than 'system' scope.
The 'site wide' config scope was meant to be per Spack
installation. A single system may have multiple Spack
installations, so was is meant for overriding the
'system' wide setting per installation.
The Spack package is OS-vendor provided. The vendor provides
pr generates a configuration which a local admin may want to
override. This can now be done from within the 'system' scope.
Previously the vendor-supplied configuration was mixed with
the 'system' scope - local modifications collided with
vendor autoconfiguration.
- Add a build-dependency package which will cause build tools
and libraries used frequently by Spack to be installed.
All these packages are recommended by the main Spack package
already. This package may be used in environments where the
installation of recommended packages is disabled by default.
-------------------------------------------------------------------
Mon May 13 13:12:22 UTC 2024 - Egbert Eich <eich@suse.com>
- Update Spack to version 0.22.0
* New features:
- Compiler dependencies:
Spack is in the process of making compilers proper dependencies.
For this, compiler dependencies are moving from `compilers.yaml`
to `packages.yaml` to make this consistent with other externals.
For this, dependency graphs will not show the compiler runtime
libraries like `gcc-runtime` or `libgfortran`.
To minimize disruption, an existing `compilers.yaml` file will
continue to work, however, users are encourage to migrate
before v0.23.
+ Packages compiled with `%gcc` now depend on a new package
`gcc-runtime`, which contains a copy of the shared compiler
runtime libraries. This enables gcc runtime libraries to be
installed and relocated when using a build cache.
When building minimal Spack-generated container images it
is no longer necessary to install libgfortran, libgomp
etc. using the system package manager.
+ Packages compiled with `%oneapi` now depend on a new package
`intel-oneapi-runtime`. This is similar to gcc-runtime.
This allows us to model library soname compatibility and
allows compilers like %oneapi to provide virtuals like sycl
(which can also be provided by standalone libraries).
+ Changes to the optimization criteria of the solver improve
the hit-rate of buildcaches by a fair amount. The solver has
more relaxed compatibility rules and will not try to strictly
match compilers or targets of reused specs. Users can still
enforce the previous strict behavior with require: sections
in packages.yaml.
Note that to enforce correct linking, Spack will not reuse
old %gcc and %oneapi specs that do not have the runtime
libraries as a dependency.
+ Spack will reuse specs built with compilers that are not
explicitly configured in `compilers.yaml`. Because we can
now keep runtime libraries in build cache, we do not require
you to also have a local configured compiler to use the
runtime libraries. This improves reuse in buildcaches and
avoids conflicts with OS updates that happen underneath
Spack.
+ Binary compatibility on `linux` is now based on the `libc`
version, instead of on the OS tag. Spack builds now detect
the host `libc` (`glibc`) and add it as an implicit external
node in the dependency graph. Binaries with a `libc` with
the same name and a version less than or equal to that of
the detected libc can be reused.
+ Each package that can provide a compiler is now detectable
using spack external find. External packages defining
compiler paths are effectively used as compilers, and spack
external find -t compiler` can be used as a substitute for
`spack compiler find. More details on this transition are
in [the docs](https://spack.readthedocs.io/en/latest/getting_started.html#manual-compiler-configuration).
- Improved spack find UI for Environments:
`spack find` in environments shows information about
* What are the roots
* Which ones are installed / not installed
* What's been added that still needs to be concretized
much more clearly. An option `--only-roots`/`-r` has been added
to `spack find` that will only show env roots, if you don't want
to look at all the installed specs.
- Improved command-line string quoting
Spack now parses specs in the CLI respecting shell quoting. Instead
of:
```
spack install zlib cflags=\"-O2 -g\"
```
you should use:
```
spack install zlib cflags="-O2 -g"
```
The old form will now result in an error.
This allows to include special characters, flags like
```
spack intall zlib ldflags='-Wl,-rpath=$ORIGIN/_libs'
```
may now be supplied. To reduce ambiguities during paring, quotes
are no longer allowed around `=` and `==`. Commands like:
```
spack install zlib cflags = "-O2 -g"
```
will result in an error.
- Revert default spack install behavior to `--reuse`
In v0.20 the default concretizer behavior was changed from
`--reuse` to `--reuse-deps`, which meant that *every* `spack
install` invocation would attempt to build a new version of
the requested package / any environment roots.
While this is a common ask for *upgrading* and for *developer
workflows*, it shouldn't be the default for a package manager.
- The `install` command now offers the three options:
* `--reuse` (default): reuse as many existing installations
as possible.
* `--reuse-deps` or `--fresh-roots`: upgrade (freshen) roots
but reuse dependencies if possible.
* --fresh: install fresh versions of requested packages
(roots) and their dependencies.
- More control over reused specs
Spack now provides better control over which packages to reuse
and how. There is a new `concretizer:reuse` config option,
which accepts the following properties:
* `roots`: true to reuse roots, false to reuse just dependencies
* `exclude`: list of constraints used to select which specs
not to reuse.
* `include`: list of constraints used to select which specs
to reuse.
* `from`: list of sources for reused specs (some combination
of `local`, `buildcache`, or `external`).
- New `conflict:` and `prefer:` syntax for package preferences
Instead of having to express conflicts and preferences just
using `require:`, these can now be expressed explicitly.
- `include_concrete` in environments
This directive allows to build on the *cncrete* contents of
another environment without changing this environment by
includeing the concrete specs from this environment's
`spack.lock`:
```
spack:
specs: []
concretizer:
unify: true
include_concrete:
- /path/to/environment1
- /path/to/environment2
```
When this environment is concretized, it will bring the
already concrete specs from `environment1` and `environment2`,
and build on top of them without changing them.
- `python-venv` isolation
Spack now insert a small `python-venv` package in between
`python` and packages that need to install Python code to
isolate Spack's build environment and shield Spack form
potential issues with an external python.
- Packages can now specify whether they may be distributed in
source or binary form.
* New commands, options and directives
- Allow packages to be pushed to build cache after install from
source.
_ `spack develop`: stage build artifacts in same root as non-dev
builds.
- Don't delete spack develop build artifacts after install.
- `spack find`: add options for local/upstream only.
- `spack logs`: print log files for packages (either partially
built or installed).
- `patch`: support reversing patches.
- `develop`: Add `-b`/`--build-directory` option to set
build_directory package attribute.
- `spack list`: add `--namesapce` / `--repo` option.
- spack gc: add options for environments and build dependencies.
- Add `--create` to `spack env activate`.
* Performance improvements
- environment.py: Fix excessive re-reads.
- ruamel yaml: Fix quadratic complexity bug.
- Refactor to improve `spec format` speed.
- Do not acquire a write lock on the env post install if no views.
- `asp.py`: fewer calls to `spec.copy()`
- `spec.py`: early return in `__str__`
- avoid `jinja2` import at startup unless needed.
* Other new features of note
- `archspec`: update to v0.2.4: bugfixes for `neoverse-v1` and
`neoverse-v2` detection.
- `spack config` get/blame: with no args, show entire config
- `spack env create <env>`: dir if dir-like
- Add handling of custom ssl certs in urllib ops.
- Add ability to rename environments.
- Add config option and compiler support to reuse across OS's.
- Support for prereleases.
- Only reuse externals when configured.
- Environments: Add support for including views.
* Binary caches
Build cache: make signed/unsigned a mirror property.
* Removals, deprecations, and syntax changes
- Remove `dpcpp` compiler and package.
- `spack load`: remove `--only` argument.
* Notable Bugfixes
- repo.py: drop deleted packages from provider cache.
- Allow `+` in module file names.
- `cmd/python`: use runpy to allow multiprocessing in scripts.
- Show extension commands with `spack -h`.
- Support environment variable expansion inside module
projections.
- Alert user to failed concretizations.
- `shell`: fix `zsh` color formatting for PS1 in environments.
- `spack mirror create --all`: include patches.
- Update `run-find-external.sh` script to address the deprecation
of `compilers.yaml`.
-------------------------------------------------------------------
Thu Apr 18 06:54:54 UTC 2024 - Egbert Eich <eich@suse.com>

76
spack.spec
View File

@ -63,7 +63,7 @@ ExclusiveArch: do_not_build
%define mypython python%{?mypyver}
Name: spack
Version: 0.21.2
Version: 0.22.0
Release: 0
Summary: Package manager for HPC systems
License: Apache-2.0 AND MIT AND Python-2.0 AND BSD-3-Clause
@ -72,7 +72,9 @@ Source0: https://github.com/spack/spack/archive/v%{version}.tar.gz#/spack
Source1: README.SUSE
Source2: spack-rpmlintrc
Source3: run-find-external.sh.in
Source4: https://en.opensuse.org/index.php?title=Spack&action=raw&ref=157522#/README-oo-wiki
# without `--header "Accept-Language: en" en.opensuse.org returns 406
#Source4: https://en.opensuse.org/index.php?title=Spack&action=raw&ref=157522#/README-oo-wiki
Source4: README-oo-wiki
# Source5 is from https://docs.python.org/3/objects.inv, but has permanent changes so using a static version
Source5: objects.inv
Source6: spack_get_libs.sh
@ -83,6 +85,7 @@ Patch6: Fix-error-during-documentation-build-due-to-recursive-module-inc
Patch7: Fix-Spinx-configuration-to-avoid-throwing-errors.patch
Patch8: Set-modules-default-to-lmod.patch
Patch9: Add-support-for-container-building-using-a-SLE-base-container.patch
Patch10: Move-site-config-scope-before-system-scope.patch
%if %{without doc}
BuildRequires: %{mypython}-urllib3
BuildRequires: fdupes
@ -107,6 +110,7 @@ Requires: make
Requires: patch
Requires: polkit
Requires: sudo
Requires: system-user-nobody
Requires: tar
Requires: unzip
Requires: xz
@ -194,6 +198,24 @@ using different compilers, options, and MPI implementations.
This package contains the info page.
%package build-dependencies
Summary: Spack Build Dependencies
Requires: bison
Requires: cmake-full
Requires: flex
Requires: libcurl-devel
Requires: libopenssl-devel
Requires: libtool
Requires: libzip-devel
Requires: ncurses-devel
Requires: xz-devel
Requires: zip
%description build-dependencies
This package provides dependencies to packages of some frequently used
build tools. If Spack finds these on the system it will not attempt to
build them.
%prep
%setup -q
%autopatch -p1
@ -246,23 +268,6 @@ grep -rl '/var/lib/spack/repos' | grep -v "cmd/list.py" | \
# spack cannot run without knowing at least the compiler, so we inject
# a dummy one
mkdir -p ${HOME}/.spack/linux/
cat > ${HOME}/.spack/linux/compilers.yaml <<EOF
compilers:
- compiler:
spec: gcc@7.5.0
paths:
cc: %{_bindir}/gcc
cxx: %{_bindir}/g++
f77: %{_bindir}/gfortran
fc: %{_bindir}/gfortran
flags: {}
operating_system: SUSE
target: x86_64
modules: []
environment:
unset: []
extra_rpaths: []
EOF
source /usr/share/spack/setup-env.sh
make man info || { cat /tmp/sphinx-err-*.log; exit 1; } #text dirhtml
rm -rf $tmpdir
@ -295,7 +300,7 @@ cp %{S:1} .
rm -rf lib/spack/spack/test
rm -rf share/spack/qa
rm -rf share/spack/logo
rm -rf var/spack/repos/builtin.mock var/spack/gpg.mock var/spack/mock_configs
rm -rf var/spack/repos/builtin.mock var/spack/gpg.mock var/spack/mock_configs var/spack/repos/duplicates.test
rm -rf lib/spack/external/ruamel/yaml/.ruamel
find . -type f -name .gitignore -delete
find . -type f -name .nojekyll -delete
@ -365,26 +370,6 @@ EOF
# make shell scripts executeable
find %{buildroot}%{_localstatedir}/lib/spack/ -type f -name \*.sh -exec chmod 755 {} \;
# Create %{_sysconfdir}/spack/compilers.yaml
mkdir -p %{buildroot}%{spack_dir}/etc/spack/
cat > %{buildroot}%{spack_dir}/etc/spack/compilers.yaml <<EOF
compilers:
- compiler:
spec: gcc@GCC_FULL_VERSION
paths:
cc: %{_bindir}/gcc-GCC_VERSION
cxx: %{_bindir}/g++-GCC_VERSION
f77: %{_bindir}/gfortran-GCC_VERSION
fc: %{_bindir}/gfortran-GCC_VERSION
flags: {}
operating_system: SUSE_VERSION
target: HOSTTYPE
modules: []
environment:
unset: []
extra_rpaths: []
EOF
# Create %{_sysconfdir}/profile.d/spack.sh
# This file properly sets MODULEPATH so lua-lmod can find the modules created by spack
mkdir -p %{buildroot}/%{_sysconfdir}/profile.d
@ -479,10 +464,6 @@ cp -r texinfo/Spack.info.gz %{buildroot}%{_infodir}
%pre -f %{name}.pre
%post
# Replace %{_sysconfdir}/spack/compilers.yaml
export GCC_VERSION=`gcc -dumpversion`
export GCC_FULL_VERSION=`gcc -dumpfullversion`
#sed -i "s@GCC_FULL_VERSION@$GCC_FULL_VERSION@" %{spack_dir}/etc/spack/modules.yaml
if [ -e /etc/os-release ] ; then
source /etc/os-release
if [ "${ID}" = "opensuse-tumbleweed" ] ; then
@ -490,12 +471,8 @@ if [ -e /etc/os-release ] ; then
else
export SPACK_NAME="${ID/-/_}${VERSION_ID/.*/}"
fi
sed -i "s@SUSE_VERSION@$SPACK_NAME@" %{spack_dir}/etc/spack/compilers.yaml
sed -i "s@SUSE_VERSION@$SPACK_NAME@g" %{_sysconfdir}/profile.d/spack.sh
fi
sed -i "s@GCC_FULL_VERSION@$GCC_FULL_VERSION@" %{spack_dir}/etc/spack/compilers.yaml
sed -i "s@GCC_VERSION@$GCC_VERSION@" %{spack_dir}/etc/spack/compilers.yaml
sed -i "s@HOSTTYPE@$HOSTTYPE@" %{spack_dir}/etc/spack/compilers.yaml
mkdir -p /opt/spack
chgrp spack /opt/spack
chmod 0775 /opt/spack
@ -521,7 +498,6 @@ chmod 0775 /opt/spack
%dir %{_prefix}/etc
%endif
%{_prefix}/etc/spack
%ghost %config %{_sysconfdir}/spack/compilers.yaml
%attr(0775, root, spack) %{_localstatedir}/lib/spack/junit-report
%attr(0775, root, spack) %{spack_dir}/opt
%attr(0775, root, spack) %{_localstatedir}/cache/spack
@ -545,6 +521,8 @@ chmod 0775 /opt/spack
%doc CHANGELOG.md NOTICE README.md
%{_datarootdir}/spack/repos
%files build-dependencies
#%{without doc}
%else