* `slurmrestd` - Remove deprecated fields from the following
`.result` from `POST /slurm/v0.0.42/job/submit`.
`.job_id`, `.step_id`, `.job_submit_user_msg` from `POST /slurm/v0.0.42/job/{job_id}`.
`.job.exclusive`, `.jobs[].exclusive` to `POST /slurm/v0.0.42/job/submit`.
`.jobs[].exclusive` from `GET /slurm/v0.0.42/job/{job_id}`.
`.jobs[].exclusive` from `GET /slurm/v0.0.42/jobs`.
`.job.oversubscribe`, `.jobs[].oversubscribe` to `POST /slurm/v0.0.42/job/submit`.
`.jobs[].oversubscribe` from `GET /slurm/v0.0.42/job/{job_id}`.
`.jobs[].oversubscribe` from `GET /slurm/v0.0.42/jobs`.
`DELETE /slurm/v0.0.40/jobs`
`DELETE /slurm/v0.0.41/jobs`
`DELETE /slurm/v0.0.42/jobs`
allocation is granted.
`job|socket|task` or `cpus|mem` per GRES.
node update whereas previously only single nodes could be
updated through `/node/<nodename>` endpoint:
`POST /slurm/v0.0.42/nodes`
partition as this is a cluster-wide option.
`REQUEST_NODE_INFO RPC`.
the db server is not reachable.
(`.jobs[].priority_by_partition`) to JSON and YAML output.
connection` error if the error was the result of an
authentication failure.
errors with the `SLURM_PROTOCOL_AUTHENTICATION_ERROR` error
code.
of `Unspecified error` if querying the following endpoints
fails:
`GET /slurm/v0.0.40/diag/`
`GET /slurm/v0.0.41/diag/`
`GET /slurm/v0.0.42/diag/` (forwarded request 1238576 from eeich)
OBS-URL: https://build.opensuse.org/request/show/1238577
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=111
- Update to version 24.11
* `slurmctld` - Reject arbitrary distribution jobs that do not
specifying a task count.
* Fix backwards compatibility of the `RESPONSE_JOB_INFO RPC`
(used by `squeue`, `scontrol show job`, etc.) with Slurm clients
version 24.05 and below. This was a regression in 24.11.0rc1.
* Do not let `slurmctld`/`slurmd` start if there are more nodes
defined in `slurm.conf` than the maximum supported amount
(64k nodes).
* `slurmctld` - Set job's exit code to 1 when a job fails with
state `JOB_NODE_FAIL`. This fixes `sbatch --wait` not being able
to exit with error code when a job fails for this reason in
some cases.
* Fix certain reservation updates requested from 23.02 clients.
* `slurmrestd` - Fix populating non-required object fields of
objects as `{}` in JSON/YAML instead of `null` causing compiled
OpenAPI clients to reject the response to
`GET /slurm/v0.0.40/jobs` due to validation failure of
`.jobs[].job_resources`.
* Fix issue where older versions of Slurm talking to a 24.11 dbd
could loose step accounting.
* Fix minor memory leaks.
* Fix bad memory reference when `xstrchr` fails to find char.
* Remove duplicate checks for a data structure.
* Fix race condition in `stepmgr` step completion handling.
* `slurm.spec` - add ability to specify patches to apply on the
command line.
* `slurm.spec` - add ability to supply extra version information.
* Fix 24.11 HA issues.
* Fix requeued jobs keeping their priority until the decay thread (forwarded request 1235783 from eeich)
OBS-URL: https://build.opensuse.org/request/show/1235784
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=109
* `slurmctld` - Reject arbitrary distribution jobs that do not
specifying a task count.
* Fix backwards compatibility of the `RESPONSE_JOB_INFO RPC`
(used by `squeue`, `scontrol show job`, etc.) with Slurm clients
version 24.05 and below. This was a regression in 24.11.0rc1.
* Do not let `slurmctld`/`slurmd` start if there are more nodes
defined in `slurm.conf` than the maximum supported amount
(64k nodes).
* `slurmctld` - Set job's exit code to 1 when a job fails with
state `JOB_NODE_FAIL`. This fixes `sbatch --wait` not being able
to exit with error code when a job fails for this reason in
some cases.
* Fix certain reservation updates requested from 23.02 clients.
* `slurmrestd` - Fix populating non-required object fields of
objects as `{}` in JSON/YAML instead of `null` causing compiled
OpenAPI clients to reject the response to
`GET /slurm/v0.0.40/jobs` due to validation failure of
`.jobs[].job_resources`.
* Fix issue where older versions of Slurm talking to a 24.11 dbd
could loose step accounting.
* Fix minor memory leaks.
* Fix bad memory reference when `xstrchr` fails to find char.
* Remove duplicate checks for a data structure.
* Fix race condition in `stepmgr` step completion handling.
* `slurm.spec` - add ability to specify patches to apply on the
command line.
* `slurm.spec` - add ability to supply extra version information.
* Fix 24.11 HA issues.
* Fix requeued jobs keeping their priority until the decay thread
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=302
- Update to version 24.05.4 & fix for CVE-2024-48936.
* Fix generic int sort functions.
* Fix user look up using possible unrealized uid in the dbd.
* `slurmrestd` - Fix regressions that allowed `slurmrestd` to
be run as SlurmUser when `SlurmUser` was not root.
* mpi/pmix fix race conditions with het jobs at step start/end
which could make srun to hang.
* Fix not showing some `SelectTypeParameters` in `scontrol show
config`.
* Avoid assert when dumping removed certain fields in JSON/YAML.
* Improve how shards are scheduled with affinity in mind.
* Fix `MaxJobsAccruePU` not being respected when `MaxJobsAccruePA`
is set in the same QOS.
* Prevent backfill from planning jobs that use overlapping
resources for the same time slot if the job's time limit is
less than `bf_resolution`.
* Fix memory leak when requesting typed gres and
`--[cpus|mem]-per-gpu`.
* Prevent backfill from breaking out due to "system state
changed" every 30 seconds if reservations use `REPLACE` or
`REPLACE_DOWN` flags.
* `slurmrestd` - Make sure that scheduler_unset parameter defaults
to true even when the following flags are also set:
`show_duplicates`, `skip_steps`, `disable_truncate_usage_time`,
`run_away_jobs`, `whole_hetjob`, `disable_whole_hetjob`,
`disable_wait_for_result`, `usage_time_as_submit_time`,
`show_batch_script`, and or `show_job_environment`. Additionaly,
always make sure show_duplicates and
`disable_truncate_usage_time` default to true when the following
flags are also set: `scheduler_unset`, `scheduled_on_submit`, (forwarded request 1220075 from eeich)
OBS-URL: https://build.opensuse.org/request/show/1220076
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=108
* Fix generic int sort functions.
* Fix user look up using possible unrealized uid in the dbd.
* `slurmrestd` - Fix regressions that allowed `slurmrestd` to
be run as SlurmUser when `SlurmUser` was not root.
* mpi/pmix fix race conditions with het jobs at step start/end
which could make srun to hang.
* Fix not showing some `SelectTypeParameters` in `scontrol show
config`.
* Avoid assert when dumping removed certain fields in JSON/YAML.
* Improve how shards are scheduled with affinity in mind.
* Fix `MaxJobsAccruePU` not being respected when `MaxJobsAccruePA`
is set in the same QOS.
* Prevent backfill from planning jobs that use overlapping
resources for the same time slot if the job's time limit is
less than `bf_resolution`.
* Fix memory leak when requesting typed gres and
`--[cpus|mem]-per-gpu`.
* Prevent backfill from breaking out due to "system state
changed" every 30 seconds if reservations use `REPLACE` or
`REPLACE_DOWN` flags.
* `slurmrestd` - Make sure that scheduler_unset parameter defaults
to true even when the following flags are also set:
`show_duplicates`, `skip_steps`, `disable_truncate_usage_time`,
`run_away_jobs`, `whole_hetjob`, `disable_whole_hetjob`,
`disable_wait_for_result`, `usage_time_as_submit_time`,
`show_batch_script`, and or `show_job_environment`. Additionaly,
always make sure show_duplicates and
`disable_truncate_usage_time` default to true when the following
flags are also set: `scheduler_unset`, `scheduled_on_submit`,
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=300
- Update to version 24.05.3
* `data_parser/v0.0.40` - Added field descriptions.
* `slurmrestd` - Avoid creating new slurmdbd connection per request
to `* /slurm/slurmctld/*/*` endpoints.
* Fix compilation issue with `switch/hpe_slingshot` plugin.
* Fix gres per task allocation with threads-per-core.
* `data_parser/v0.0.41` - Added field descriptions.
* `slurmrestd` - Change back generated OpenAPI schema for
`DELETE /slurm/v0.0.40/jobs/` to `RequestBody` instead of using
parameters for request. `slurmrestd` will continue accept endpoint
requests via `RequestBody` or HTTP query.
* `topology/tree` - Fix issues with switch distance optimization.
* Fix potential segfault of secondary `slurmctld` when falling back
to the primary when running with a `JobComp` plugin.
* Enable `--json`/`--yaml=v0.0.39` options on client commands to
dump data using data_parser/v0.0.39 instead or outputting nothing.
* `switch/hpe_slingshot` - Fix issue that could result in a 0 length
state file.
* Fix unnecessary message protocol downgrade for unregistered nodes.
* Fix unnecessarily packing alias addrs when terminating jobs with
a mix of non-cloud/dynamic nodes and powered down cloud/dynamic
nodes.
* `accounting_storage/mysql` - Fix issue when deleting a qos that
could remove too many commas from the qos and/or delta_qos fields
of the assoc table.
* `slurmctld` - Fix memory leak when using RestrictedCoresPerGPU.
* Fix allowing access to reservations without `MaxStartDelay` set.
* Fix regression introduced in 24.05.0rc1 breaking
`srun --send-libs` parsing.
* Fix slurmd vsize memory leak when using job submission/allocation
OBS-URL: https://build.opensuse.org/request/show/1208086
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=106
* `data_parser/v0.0.40` - Added field descriptions.
* `slurmrestd` - Avoid creating new slurmdbd connection per request
to `* /slurm/slurmctld/*/*` endpoints.
* Fix compilation issue with `switch/hpe_slingshot` plugin.
* Fix gres per task allocation with threads-per-core.
* `data_parser/v0.0.41` - Added field descriptions.
* `slurmrestd` - Change back generated OpenAPI schema for
`DELETE /slurm/v0.0.40/jobs/` to `RequestBody` instead of using
parameters for request. `slurmrestd` will continue accept endpoint
requests via `RequestBody` or HTTP query.
* `topology/tree` - Fix issues with switch distance optimization.
* Fix potential segfault of secondary `slurmctld` when falling back
to the primary when running with a `JobComp` plugin.
* Enable `--json`/`--yaml=v0.0.39` options on client commands to
dump data using data_parser/v0.0.39 instead or outputting nothing.
* `switch/hpe_slingshot` - Fix issue that could result in a 0 length
state file.
* Fix unnecessary message protocol downgrade for unregistered nodes.
* Fix unnecessarily packing alias addrs when terminating jobs with
a mix of non-cloud/dynamic nodes and powered down cloud/dynamic
nodes.
* `accounting_storage/mysql` - Fix issue when deleting a qos that
could remove too many commas from the qos and/or delta_qos fields
of the assoc table.
* `slurmctld` - Fix memory leak when using RestrictedCoresPerGPU.
* Fix allowing access to reservations without `MaxStartDelay` set.
* Fix regression introduced in 24.05.0rc1 breaking
`srun --send-libs` parsing.
* Fix slurmd vsize memory leak when using job submission/allocation
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=295
- IMPORTANT NOTES:
If using the slurmdbd (Slurm DataBase Daemon) you must update
this first. NOTE: If using a backup DBD you must start the
primary first to do any database conversion, the backup will not
start until this has happened. The 24.05 slurmdbd will work
with Slurm daemons of version 23.02 and above. You will not
need to update all clusters at the same time, but it is very
important to update slurmdbd first and having it running before
updating any other clusters making use of it.
- HIGHLIGHTS
* Federation - allow client command operation when slurmdbd is
unavailable.
* burst_buffer/lua - Added two new hooks: slurm_bb_test_data_in
and slurm_bb_test_data_out. The syntax and use of the new hooks
are documented in etc/burst_buffer.lua.example. These are
required to exist. slurmctld now checks on startup if the
burst_buffer.lua script loads and contains all required hooks;
slurmctld will exit with a fatal error if this is not
successful. Added PollInterval to burst_buffer.conf. Removed
the arbitrary limit of 512 copies of the script running
simultaneously.
* Add QOS limit MaxTRESRunMinsPerAccount.
* Add QOS limit MaxTRESRunMinsPerUser.
* Add ELIGIBLE environment variable to jobcomp/script plugin.
* Always use the QOS name for SLURM_JOB_QOS environment variables.
Previously the batch environment would use the description field,
which was usually equivalent to the name.
* cgroup/v2 - Require dbus-1 version >= 1.11.16.
* Allow NodeSet names to be used in SuspendExcNodes.
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=294
- removed Keep-logs-of-skipped-test-when-running-test-cases-sequentially.patch
as incoperated upstream
* Changes in Slurm 23.02.5
* Add the JobId to debug() messages indicating when cpus_per_task/mem_per_cpu
or pn_min_cpus are being automatically adjusted.
* Fix regression in 23.02.2 that caused slurmctld -R to crash on startup if
a node features plugin is configured.
* Fix and prevent reoccurring reservations from overlapping.
* job_container/tmpfs - Avoid attempts to share BasePath between nodes.
* Change the log message warning for rate limited users from verbose to info.
* With CR_Cpu_Memory, fix node selection for jobs that request gres and
*-mem-per-cpu.
* Fix a regression from 22.05.7 in which some jobs were allocated too few
nodes, thus overcommitting cpus to some tasks.
* Fix a job being stuck in the completing state if the job ends while the
primary controller is down or unresponsive and the backup controller has
not yet taken over.
* Fix slurmctld segfault when a node registers with a configured CpuSpecList
while slurmctld configuration has the node without CpuSpecList.
* Fix cloud nodes getting stuck in POWERED_DOWN+NO_RESPOND state after not
registering by ResumeTimeout.
* slurmstepd - Avoid cleanup of config.json-less containers spooldir getting
skipped.
* slurmstepd - Cleanup per task generated environment for containers in
spooldir.
* Fix scontrol segfault when 'completing' command requested repeatedly in
interactive mode.
* Properly handle a race condition between bind() and listen() calls in the
network stack when running with SrunPortRange set.
* Federation - Fix revoked jobs being returned regardless of the -a/--all
OBS-URL: https://build.opensuse.org/request/show/1161499
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=292
- Update to version 23.11.03
* slurmrestd - Reject single http query with multiple path
requests.
* Fix launching Singularity v4.x containers with
`srun --container` by setting .process.terminal to true in
generated `config.json` when step has pseudoterminal (`--pty`)
requested.
* Fix loading in `dyanmic/cloud` node jobs after `net_cred`
expired.
* Fix cgroup null path error on `slurmd/slurmstepd` tear down.
* `data_parser/v0.0.40` - Prevent failure if accounting is
disabled, instead issue a warning if needed data from the
database can not be retrieved.
* `openapi/slurmctld` - Prevent failure if accounting is disabled.
* Prevent `slurmscriptd` processing delays from blocking other
threads in `slurmctld` while trying to launch various scripts.
This is additional work for a fix in 23.02.6.
* Fix memory leak when receiving alias addrs from controller.
* `scontrol` - Accept `scontrol token lifespan=infinite` to
create tokens that effectively do not expire.
* Avoid errors when Slurmdb accounting disabled when `--json` or
`--yaml` is invoked with CLI commands and `slurmrestd`. Add
warnings when query would have populated data from Slurmdb
instead of errors.
* Fix `slurmctld` memory leak when running job with
`--tres-per-task=gres:shard:#`
* Fix backfill trying to start jobs outside of backfill window.
* Fix oversubscription on partitions with `PreemptMode=OFF`.
* Preserve node reason on power up if the node is downed
or drained. (forwarded request 1150524 from eeich)
OBS-URL: https://build.opensuse.org/request/show/1151965
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=104
- Update to version 23.11.03
* slurmrestd - Reject single http query with multiple path
requests.
* Fix launching Singularity v4.x containers with
`srun --container` by setting .process.terminal to true in
generated `config.json` when step has pseudoterminal (`--pty`)
requested.
* Fix loading in `dyanmic/cloud` node jobs after `net_cred`
expired.
* Fix cgroup null path error on `slurmd/slurmstepd` tear down.
* `data_parser/v0.0.40` - Prevent failure if accounting is
disabled, instead issue a warning if needed data from the
database can not be retrieved.
* `openapi/slurmctld` - Prevent failure if accounting is disabled.
* Prevent `slurmscriptd` processing delays from blocking other
threads in `slurmctld` while trying to launch various scripts.
This is additional work for a fix in 23.02.6.
* Fix memory leak when receiving alias addrs from controller.
* `scontrol` - Accept `scontrol token lifespan=infinite` to
create tokens that effectively do not expire.
* Avoid errors when Slurmdb accounting disabled when `--json` or
`--yaml` is invoked with CLI commands and `slurmrestd`. Add
warnings when query would have populated data from Slurmdb
instead of errors.
* Fix `slurmctld` memory leak when running job with
`--tres-per-task=gres:shard:#`
* Fix backfill trying to start jobs outside of backfill window.
* Fix oversubscription on partitions with `PreemptMode=OFF`.
* Preserve node reason on power up if the node is downed
or drained.
OBS-URL: https://build.opensuse.org/request/show/1150524
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=289
- Update to 23.11.1 with following major improvements and fixing
CVE-2023-49933, CVE-2023-49934, CVE-2023-49935, CVE-2023-49936
and CVE-2023-49937
* Substantially overhauled the SlurmDBD association management
code. For clusters updated to 23.11, account and user
additions or removals are significantly faster than in prior
releases.
* Overhauled `scontrol reconfigure` to prevent configuration
mistakes from disabling slurmctld and slurmd. Instead, an
error will be returned, and the running configuration will
persist. This does require updates to the systemd service
files to use the `--systemd` option to `slurmctld` and `slurmd`.
* Added a new internal `auth/cred` plugin - `auth/slurm`. This
builds off the prior `auth/jwt` model, and permits operation
of the `slurmdbd` and `slurmctld` without access to full
directory information with a suitable configuration.
* Added a new `--external-launcher` option to `srun`, which is
automatically set by common MPI launcher implementations and
ensures processes using those non-srun launchers have full
access to all resources allocated on each node.
* Reworked the dynamic/cloud modes of operation to allow for
"fanout" - where Slurm communication can be automatically
offloaded to compute nodes for increased cluster scalability.
* Overhauled and extended the Reservation subsystem to allow
for most of the same resource requirements as are placed on
the job. Notably, this permits reservations to now reserve
GRES directly.
- Details of changes:
* Fix `scontrol update job=... TimeLimit+=/-=` when used with a
raw JobId of job array element.
OBS-URL: https://build.opensuse.org/request/show/1141442
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=103
and CVE-2023-49937
* Substantially overhauled the SlurmDBD association management
code. For clusters updated to 23.11, account and user
additions or removals are significantly faster than in prior
releases.
* Overhauled `scontrol reconfigure` to prevent configuration
mistakes from disabling slurmctld and slurmd. Instead, an
error will be returned, and the running configuration will
persist. This does require updates to the systemd service
files to use the `--systemd` option to `slurmctld` and `slurmd`.
* Added a new internal `auth/cred` plugin - `auth/slurm`. This
builds off the prior `auth/jwt` model, and permits operation
of the `slurmdbd` and `slurmctld` without access to full
directory information with a suitable configuration.
* Added a new `--external-launcher` option to `srun`, which is
automatically set by common MPI launcher implementations and
ensures processes using those non-srun launchers have full
access to all resources allocated on each node.
* Reworked the dynamic/cloud modes of operation to allow for
"fanout" - where Slurm communication can be automatically
offloaded to compute nodes for increased cluster scalability.
* Overhauled and extended the Reservation subsystem to allow
for most of the same resource requirements as are placed on
the job. Notably, this permits reservations to now reserve
GRES directly.
* Fix `scontrol update job=... TimeLimit+=/-=` when used with a
raw JobId of job array element.
* Reject `TimeLimit` increment/decrement when called on job with
`TimeLimit=UNLIMITED`.
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=285
- Update to 23.11.1 with following major improvements and fixing
CVE-2023-49933, CVE-2023-49934, CVE-2023-49935, CVE-2023-49936 and
CVE-2023-49937
* Substantially overhauled the SlurmDBD association management code. For
clusters updated to 23.11, account and user additions or removals are
significantly faster than in prior releases.
* Overhauled 'scontrol reconfigure' to prevent configuration mistakes from
disabling slurmctld and slurmd. Instead, an error will be returned, and the
running configuration will persist. This does require updates to the
systemd service files to use the --systemd option to slurmctld and slurmd.
* Added a new internal auth/cred plugin - "auth/slurm". This builds off the
prior auth/jwt model, and permits operation of the slurmdbd and slurmctld
without access to full directory information with a suitable configuration.
* Added a new --external-launcher option to srun, which is automatically set
by common MPI launcher implementations and ensures processes using those
non-srun launchers have full access to all resources allocated on each
node.
* Reworked the dynamic/cloud modes of operation to allow for "fanout" - where
Slurm communication can be automatically offloaded to compute nodes for
increased cluster scalability.
Added initial official Debian packaging support.
* Overhauled and extended the Reservation subsystem to allow for most of the
same resource requirements as are placed on the job. Notably, this permits
reservations to now reserve GRES directly.
- Details of changes:
* Fix scontrol update job=... TimeLimit+=/-= when used with a raw JobId of job
array element.
* Reject TimeLimit increment/decrement when called on job with
TimeLimit=UNLIMITED.
* Fix issue with requesting a job with *licenses as well as
OBS-URL: https://build.opensuse.org/request/show/1138332
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=284
- Update to 23.02.6 to fix (CVE-2023-49933 - bsc#1218046, CVE-2023-49935 -
bsc#1218049, CVE-2023-49936 - bsc#1218050, CVE-2023-49937 - bsc#1218051,
CVE-2023-49938 - bsc#1218053)
* Security Fixes:
+ Add `JobAcctGatherParams=DisableGPUAcct` to disable gpu accounting.
+ `acct_gather_energy/ipmi` - Improve logging of DCMI issues.
+ `gpu/oneapi` - Add support for new env vars `ZE_FLAT_DEVICE_HIERARCHY`
and `ZE_ENABLE_PCI_ID_DEVICE_ORDER`.
+ `data_parser/v0.0.39` - skip empty string when parsing QOS ids.
+ Remove error message from `assoc_mgr_update_assocs` when purposefully
resetting the default QOS.
* Bug Fixes:
+ `libslurm_nss` - Avoid causing glibc to assert due to an unexpected
return from slurm_nss due to an error during lookup.
+ Fix job requests with `--tres-per-task` sometimes resulting in bad
allocations that cannot run subsequent job steps.
+ Fix issue with `slurmd` where `srun` fails to be warned when a node
prolog script runs beyond `MsgTimeout` set in `slurm.conf`.
+ `gres/shard` - Fix plugin functions to have matching parameter orders.
+ `gpu/nvml` - Fix issue that resulted in the wrong MIG devices being
constrained to a job
+ `gpu/nvml` - Fix linking issue with MIGs that prevented multiple MIGs
being used in a single job for certain MIG configurations
+ Fix file descriptor leak in slurmd when using `acct_gather_energy/ipmi`
with DCMI devices.
+ `sview` - avoid crash when job has a node list string > 49 characters.
+ Prevent `slurmctld` crash during reconfigure when packing job start
messages.
+ Preserve reason uid on reconfig.
+ Update node reason with updated `INVAL` state reason if different from (forwarded request 1136624 from eeich)
OBS-URL: https://build.opensuse.org/request/show/1137045
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=102
- Update to 23.02.6 to fix (CVE-2023-49933 - bsc#1218046, CVE-2023-49935 -
bsc#1218049, CVE-2023-49936 - bsc#1218050, CVE-2023-49937 - bsc#1218051,
CVE-2023-49938 - bsc#1218053)
* Security Fixes:
+ Add `JobAcctGatherParams=DisableGPUAcct` to disable gpu accounting.
+ `acct_gather_energy/ipmi` - Improve logging of DCMI issues.
+ `gpu/oneapi` - Add support for new env vars `ZE_FLAT_DEVICE_HIERARCHY`
and `ZE_ENABLE_PCI_ID_DEVICE_ORDER`.
+ `data_parser/v0.0.39` - skip empty string when parsing QOS ids.
+ Remove error message from `assoc_mgr_update_assocs` when purposefully
resetting the default QOS.
* Bug Fixes:
+ `libslurm_nss` - Avoid causing glibc to assert due to an unexpected
return from slurm_nss due to an error during lookup.
+ Fix job requests with `--tres-per-task` sometimes resulting in bad
allocations that cannot run subsequent job steps.
+ Fix issue with `slurmd` where `srun` fails to be warned when a node
prolog script runs beyond `MsgTimeout` set in `slurm.conf`.
+ `gres/shard` - Fix plugin functions to have matching parameter orders.
+ `gpu/nvml` - Fix issue that resulted in the wrong MIG devices being
constrained to a job
+ `gpu/nvml` - Fix linking issue with MIGs that prevented multiple MIGs
being used in a single job for certain MIG configurations
+ Fix file descriptor leak in slurmd when using `acct_gather_energy/ipmi`
with DCMI devices.
+ `sview` - avoid crash when job has a node list string > 49 characters.
+ Prevent `slurmctld` crash during reconfigure when packing job start
messages.
+ Preserve reason uid on reconfig.
+ Update node reason with updated `INVAL` state reason if different from
OBS-URL: https://build.opensuse.org/request/show/1136624
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=282
- Explicitly create an Obsoletes: entry for each package version
that is obsoleted by the present version. These are all published
versions of the last two major releases as well as all minor
versions of the present release lower than the current one
(bsc#1216869 2nd part).
This prevents the current version to upgrade a old Slurm version
for which no upgrade path exists.
OBS-URL: https://build.opensuse.org/request/show/1129638
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=279
- update to 23.02.6 to fix (CVE-2023-41914)
* Removed Fix-test-32.8.patch as fixed upstream
* Bug Fixes:
+ Fix `CpusPerTres=` not upgreadable with scontrol update
+ Fix unintentional gres removal when validating the gres job state.
+ Fix `--without-hpe-slingshot` configure option.
+ Fix cgroup v2 memory calculations when transparent huge pages are used.
+ Fix parsing of `sgather --timeout` option.
+ Fix regression from 22.05.0 that caused `srun --cpu-bind "=verbose"`
and `"=v"` options give different CPU bind masks.
+ Fix "_find_node_record: lookup failure for node" error message appearing
for all dynamic nodes during reconfigure.
+ Avoid segfault if loading serializer plugin fails.
+ `slurmrestd` - Correct OpenAPI format for `GET /slurm/v0.0.39/licenses`.
+ `slurmrestd` - Correct OpenAPI format for
`GET /slurm/v0.0.39/job/{job_id}`.
+ `slurmrestd` - Change format to multiple fields in
'GET /slurmdb/v0.0.39/assocations` and `GET /slurmdb/v0.0.39/qos` to
handle infinite and unset states.
+ When a node fails in a job with `--no-kill`, preserve the extern step on the
remaining nodes to avoid breaking features that rely on the extern step
such as `pam_slurm_adopt`, `x11`, and `job_container/tmpfs`.
+ `auth/jwt` - Ignore `x5c` field in JWKS files.
+ `auth/jwt` - Treat 'alg' field as optional in JWKS files.
+ Allow job_desc.selinux_context to be read from the job_submit.lua script.
+ Skip check in slurmstepd that causes a large number of errors in the
munge log: "Unauthorized credential for client UID=0 GID=0".
This error will still appear on `slurmd`/`slurmctld`/`slurmdbd` start up
and is not a cause for concern.
+ `slurmctld` - Allow startup with zero partitions.
OBS-URL: https://build.opensuse.org/request/show/1117163
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=96
* Bug Fixes:
+ Fix CpusPerTres= not upgreadable with scontrol update
+ Fix unintentional gres removal when validating the gres job state.
+ Fix --without-hpe-slingshot configure option.
+ Fix cgroup v2 memory calculations when transparent huge pages are used.
+ Fix parsing of sgather --timeout option.
+ Fix regression from 22.05.0 that caused srun --cpu-bind "=verbose" and "=v"
options give different CPU bind masks.
+ Fix "_find_node_record: lookup failure for node" error message appearing
for all dynamic nodes during reconfigure.
+ Avoid segfault if loading serializer plugin fails.
+ slurmrestd - Correct OpenAPI format for 'GET /slurm/v0.0.39/licenses'.
+ slurmrestd - Correct OpenAPI format for 'GET /slurm/v0.0.39/job/{job_id}'.
+ slurmrestd - Change format to multiple fields in 'GET
/slurmdb/v0.0.39/assocations' and 'GET /slurmdb/v0.0.39/qos' to handle
infinite and unset states.
+ When a node fails in a job with --no-kill, preserve the extern step on the
remaining nodes to avoid breaking features that rely on the extern step
such as pam_slurm_adopt, x11, and job_container/tmpfs.
+ auth/jwt - Ignore 'x5c' field in JWKS files.
+ auth/jwt - Treat 'alg' field as optional in JWKS files.
+ Allow job_desc.selinux_context to be read from the job_submit.lua script.
+ Skip check in slurmstepd that causes a large number of errors in the munge
log: "Unauthorized credential for client UID=0 GID=0". This error will
still appear on slurmd/slurmctld/slurmdbd start up and is not a cause for
concern.
+ slurmctld - Allow startup with zero partitions.
+ Fix some mig profile names in slurm not matching nvidia mig profiles.
+ Prevent slurmscriptd processing delays from blocking other threads in
slurmctld while trying to launch {Prolog|Epilog}Slurmctld.
OBS-URL: https://build.opensuse.org/request/show/1117145
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=268
- Updated to version 23.02.5 with the following changes:
* Bug Fixes:
+ Revert a change in 23.02 where `SLURM_NTASKS` was no longer set in the
job's environment when `--ntasks-per-node` was requested.
The method that is is being set, however, is different and should be more
accurate in more situations.
+ Change pmi2 plugin to honor the `SrunPortRange` option. This matches the
new behavior of the pmix plugin in 23.02.0. Note that neither of these
plugins makes use of the `MpiParams=ports=` option, and previously
were only limited by the systems ephemeral port range.
+ Fix regression in 23.02.2 that caused slurmctld -R to crash on startup if
a node features plugin is configured.
+ Fix and prevent reoccurring reservations from overlapping.
+ `job_container/tmpfs` - Avoid attempts to share BasePath between nodes.
+ With `CR_Cpu_Memory`, fix node selection for jobs that request gres and
`--mem-per-cpu`.
+ Fix a regression from 22.05.7 in which some jobs were allocated too few
nodes, thus overcommitting cpus to some tasks.
+ Fix a job being stuck in the completing state if the job ends while the
primary controller is down or unresponsive and the backup controller has
not yet taken over.
+ Fix `slurmctld` segfault when a node registers with a configured
`CpuSpecList` while `slurmctld` configuration has the node without
`CpuSpecList`.
+ Fix cloud nodes getting stuck in `POWERED_DOWN+NO_RESPOND` state after
not registering by `ResumeTimeout`.
+ `slurmstepd` - Avoid cleanup of `config.json-less` containers spooldir
getting skipped.
+ Fix scontrol segfault when 'completing' command requested repeatedly in
interactive mode.
OBS-URL: https://build.opensuse.org/request/show/1111943
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=95
features with the `|` operator, which could prevent jobs from
+ `node_features/helpers` - Fix inconsistent handling of `&` and `|`,
instead of just the current set. E.g. `foo|bar&baz` was interpreted
`{foo} or {bar,baz}`.
tasks fewer than GPUs, which resulted in incorrectly rejecting these
jobs.
+ `slurmrestd` - For `GET /slurm/v0.0.39/node[s]`, change format of
node's energy field `current_watts` to a dictionary to account for
+ `slurmrestd` - For `GET /slurm/v0.0.39/qos`, change format of QOS's
+ slurmrestd - For `GET /slurm/v0.0.39/job[s]`, the 'return code'
`GET /slurmdb/v0.0.39/jobs` from slurmrestd.
were present in the log: `error: Attempt to change gres/gpu Count`.
+ Hold the job with `(Reservation ... invalid)` state reason if the
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=265
* Bug Fixes:
+ Revert a change in 23.02 where `SLURM_NTASKS` was no longer set in the
job's environment when `--ntasks-per-node` was requested.
The method that is is being set, however, is different and should be more
accurate in more situations.
+ Change pmi2 plugin to honor the `SrunPortRange` option. This matches the
new behavior of the pmix plugin in 23.02.0. Note that neither of these
plugins makes use of the "`MpiParams=ports=`" option, and previously
were only limited by the systems ephemeral port range.
+ Fix regression in 23.02.2 that caused slurmctld -R to crash on startup if
a node features plugin is configured.
+ Fix and prevent reoccurring reservations from overlapping.
+ `job_container/tmpfs` - Avoid attempts to share BasePath between nodes.
+ With `CR_Cpu_Memory`, fix node selection for jobs that request gres and
`--mem-per-cpu`.
+ Fix a regression from 22.05.7 in which some jobs were allocated too few
nodes, thus overcommitting cpus to some tasks.
+ Fix a job being stuck in the completing state if the job ends while the
primary controller is down or unresponsive and the backup controller has
not yet taken over.
+ Fix `slurmctld` segfault when a node registers with a configured
`CpuSpecList` while `slurmctld` configuration has the node without
`CpuSpecList`.
+ Fix cloud nodes getting stuck in `POWERED_DOWN+NO_RESPOND` state after
not registering by `ResumeTimeout`.
+ `slurmstepd` - Avoid cleanup of `config.json-less` containers spooldir
getting skipped.
+ Fix scontrol segfault when 'completing' command requested repeatedly in
interactive mode.
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=264
- Updated to 23.02.4 with the following changes:
* Bug Fixes:
+ Fix main scheduler loop not starting after a failover to backup
controller. Avoid slurmctld segfault when specifying
`AccountingStorageExternalHost` (bsc#1214983).
+ Fix sbatch return code when `--wait` is requested on a job array.
+ Fix collected `GPUUtilization` values for `acct_gather_profile` plugins.
+ Fix `slurmrestd` handling of job hold/release operations.
+ Fix step running indefinitely when slurmctld takes more than
`MessageTimeout` to respond. Now, `slurmctld` will cancel the step when
detected, preventing following steps from getting stuck waiting for
resources to be released.
+ Fix regression to make `job_desc.min_cpus` accurate again in `job_submit`
when requesting a job with `--ntasks-per-node`.
+ Fix handling of `ArrayTaskThrottle` in backfill.
+ Fix regression in 23.02.2 when checking gres state on `slurmctld`
startup or reconfigure. Gres changes in the configuration were not
updated on slurmctld startup. On startup or reconfigure, these messages
were present in the log: `"error: Attempt to change gres/gpu Count`".
+ Fix potential double count of gres when dealing with limits.
+ Fix `slurmstepd` segfault when `ContainerPath` is not set in `oci.conf`
+ Fixed an issue where jobs requesting licenses were incorrectly rejected.
+ `scrontab` - Fix cutting off the final character of quoted variables.
+ `smail` - Fix issues where e-mails at job completion were not being sent.
+ `scontrol/slurmctld` - fix comma parsing when updating a reservation's
nodes.
+ Fix `--gpu-bind=single binding` tasks to wrong gpus, leading to some gpus
having more tasks than they should and other gpus being unused.
+ Fix regression in 23.02 that causes slurmstepd to crash when `srun`
requests more than `TreeWidth` nodes in a step and uses the pmi2 or
OBS-URL: https://build.opensuse.org/request/show/1110259
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=93
* Bug Fixes:
+ Fix main scheduler loop not starting after a failover to backup
controller. Avoid slurmctld segfault when specifying
`AccountingStorageExternalHost` (bsc#1214983).
+ Fix sbatch return code when `--wait` is requested on a job array.
+ Fix collected `GPUUtilization` values for `acct_gather_profile` plugins.
+ Fix `slurmrestd` handling of job hold/release operations.
+ Fix step running indefinitely when slurmctld takes more than
`MessageTimeout` to respond. Now, `slurmctld` will cancel the step when
detected, preventing following steps from getting stuck waiting for
resources to be released.
+ Fix regression to make `job_desc.min_cpus` accurate again in `job_submit`
when requesting a job with `--ntasks-per-node`.
+ Fix handling of `ArrayTaskThrottle` in backfill.
+ Fix regression in 23.02.2 when checking gres state on `slurmctld`
startup or reconfigure. Gres changes in the configuration were not
updated on slurmctld startup. On startup or reconfigure, these messages
were present in the log: `"error: Attempt to change gres/gpu Count`".
+ Fix potential double count of gres when dealing with limits.
+ Fix `slurmstepd` segfault when `ContainerPath` is not set in `oci.conf`
+ Fixed an issue where jobs requesting licenses were incorrectly rejected.
+ `scrontab` - Fix cutting off the final character of quoted variables.
+ `smail` - Fix issues where e-mails at job completion were not being sent.
+ `scontrol/slurmctld` - fix comma parsing when updating a reservation's
nodes.
+ Fix `--gpu-bind=single binding` tasks to wrong gpus, leading to some gpus
having more tasks than they should and other gpus being unused.
+ Fix regression in 23.02 that causes slurmstepd to crash when `srun`
requests more than `TreeWidth` nodes in a step and uses the pmi2 or
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=260
- Fixes since 23.02.03:
Highlights:
* Fix main scheduler loop not starting after a failover to backup controller.
* Avoid slurmctld segfault when specifying `AccountingStorageExternalHost`
(bsc#1214983).
Other:
* Fix sbatch return code when `--wait` is requested on a job array.
* Fix collected `GPUUtilization` values for `acct_gather_profile` plugins.
* Fix `slurmrestd` handling of job hold/release operations.
* Make spank `S_JOB_ARGV` item value hold the requested command `argv`
instead of the `srun --bcast` value when `--bcast` requested (only in local
context).
* Fix step running indefinitely when slurmctld takes more than
`MessageTimeout` to respond. Now, slurmctld will cancel the step when
detected, preventing following steps from getting stuck waiting for
resources to be released.
* Fix regression to make `job_desc.min_cpus` accurate again in job_submit when
requesting a job with `--ntasks-per-node`.
* Fix handling of `ArrayTaskThrottle` in backfill.
* Fix regression in 23.02.2 when checking gres state on `slurmctld` startup or
reconfigure. Gres changes in the configuration were not updated on slurmctld
startup. On startup or reconfigure, these messages were present in the log:
`"error: Attempt to change gres/gpu Count`".
* Fix potential double count of gres when dealing with limits.
* Fix slurmstepd segfault when ContainerPath is not set in `oci.conf`
* Fixed an issue where jobs requesting licenses were incorrectly rejected.
* `scrontab` - Fix cutting off the final character of quoted variables.
* `smail` - Fix issues where e-mails at job completion were not being sent.
* `scontrol/slurmctld` - fix comma parsing when updating a reservation's
nodes.
OBS-URL: https://build.opensuse.org/request/show/1109308
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=92
Highlights:
* Fix main scheduler loop not starting after a failover to backup controller.
* Avoid slurmctld segfault when specifying `AccountingStorageExternalHost`
(bsc#1214983).
Other:
* Fix sbatch return code when `--wait` is requested on a job array.
* Fix collected `GPUUtilization` values for `acct_gather_profile` plugins.
* Fix `slurmrestd` handling of job hold/release operations.
* Make spank `S_JOB_ARGV` item value hold the requested command `argv`
instead of the `srun --bcast` value when `--bcast` requested (only in local
context).
* Fix step running indefinitely when slurmctld takes more than
`MessageTimeout` to respond. Now, slurmctld will cancel the step when
detected, preventing following steps from getting stuck waiting for
resources to be released.
* Fix regression to make `job_desc.min_cpus` accurate again in job_submit when
requesting a job with `--ntasks-per-node`.
* Fix handling of `ArrayTaskThrottle` in backfill.
* Fix regression in 23.02.2 when checking gres state on `slurmctld` startup or
reconfigure. Gres changes in the configuration were not updated on slurmctld
startup. On startup or reconfigure, these messages were present in the log:
`"error: Attempt to change gres/gpu Count`".
* Fix potential double count of gres when dealing with limits.
* Fix slurmstepd segfault when ContainerPath is not set in `oci.conf`
* Fixed an issue where jobs requesting licenses were incorrectly rejected.
* `scrontab` - Fix cutting off the final character of quoted variables.
* `smail` - Fix issues where e-mails at job completion were not being sent.
* `scontrol/slurmctld` - fix comma parsing when updating a reservation's
nodes.
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=258
- updated to 23.02.04 which includes following changes:
* fixing the main scheduler loop not starting on the backup controller after
a failover event, a segfault when attempting to use
* AccountingStorageExternalHost, and an issue where steps could continue
running indefinitely if the slurmctld takes too long to respond (bsc#1214983)
* include a fix for a potential slurmctld crashes when the backup slurmctld
takes over.
* This also fixes some issues when using older versions of the command line
tools with a 23.02 controller.
* srun/sbatch/salloc - In order to support user namespaces, process user and
group ids are no longer used unless explicitly requested as an argument and
are left as nobody(99) by default. Any cli_filters or SPANK plugins need to
ignore any uid or gid that equal SLURM_AUTH_NOBODY (99). User and group ids
are now resolved by the active auth plugin. To determine the actual job uid
or gid you should use the RESPONSE_RESOURCE_ALLOCATION RPC.
- removed Fix-test-3.13.patch as fixed upstream
- removed Fix-test-38.11.patch as test changed upstream (forwarded request 1109009 from mslacken)
OBS-URL: https://build.opensuse.org/request/show/1109029
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=91
- updated to 23.02.04 which includes following changes:
* fixing the main scheduler loop not starting on the backup controller after
a failover event, a segfault when attempting to use
* AccountingStorageExternalHost, and an issue where steps could continue
running indefinitely if the slurmctld takes too long to respond (bsc#1214983)
* include a fix for a potential slurmctld crashes when the backup slurmctld
takes over.
* This also fixes some issues when using older versions of the command line
tools with a 23.02 controller.
* srun/sbatch/salloc - In order to support user namespaces, process user and
group ids are no longer used unless explicitly requested as an argument and
are left as nobody(99) by default. Any cli_filters or SPANK plugins need to
ignore any uid or gid that equal SLURM_AUTH_NOBODY (99). User and group ids
are now resolved by the active auth plugin. To determine the actual job uid
or gid you should use the RESPONSE_RESOURCE_ALLOCATION RPC.
- removed Fix-test-3.13.patch as fixed upstream
- removed Fix-test-38.11.patch as test changed upstream
OBS-URL: https://build.opensuse.org/request/show/1109009
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=256
- updated to 23.02.02 which includes a number of fixes to Slurm stability
* Includes a fix for a regression in 23.02 that caused openmpi mpirun to fail
to launch tasks.
* It also includes two functional changes: Don't update the cron job tasks if
the whole crontab file is left untouched after opening it with scrontab -e
* Sort dynamic nodes and include them in topology after scontrol reconfigure
or a slurmctld restart.
OBS-URL: https://build.opensuse.org/request/show/1085668
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=254
- Web-configurator: changed presets to SUSE defaults.
- If %_restart_on_update is no longer defined replace by own
macro.
- Marked slurm-openlava, slurm-seff and slurm-sjstat noarch.
- rpmlint:
* dropped some rpmlint filters which are no longer relevant.
* added/refreshed filters. For Details, see rpmlintrc.
- Remove workaround to fix the restart issue in an Slurm package
described in bsc#1088693.
The Slurm version in this package as 16.05. Any attempt to
directly migrate to the current version is bound to fail
anyway.
- Now require slurm-munge if munge authentication is installed.
OBS-URL: https://build.opensuse.org/request/show/1083466
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=89
- Web-configurator: changed presets to SUSE defaults.
- If %_restart_on_update is no longer defined replace by own
macro.
- Marked slurm-openlava, slurm-seff and slurm-sjstat noarch.
- rpmlint:
* dropped some rpmlint filters which are no longer relevant.
* added/refreshed filters. For Details, see rpmlintrc.
- Remove workaround to fix the restart issue in an Slurm package
described in bsc#1088693.
The Slurm version in this package as 16.05. Any attempt to
directly migrate to the current version is bound to fail
anyway.
OBS-URL: https://build.opensuse.org/request/show/1082770
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=251
- updated to 23.02.1 with the following changes:
* job_container/tmpfs - cleanup job container even if namespace mount is
already unmounted.
* openapi/dbv0.0.38 - Fix not displaying an error when updating QOS or
associations fails.
* Fix nodes remaining as PLANNED after slurmctld save state recovery.
* Add cgroup.conf EnableControllers option for cgroup/v2.
* Get correct cgroup root to allow slurmd to run in containers like Docker.
* slurmctld - add missing PrivateData=jobs check to step ContainerID lookup
requests originated from 'scontrol show step container-id=<id>' or certain
scrun operations when container state can't be directly queried.
* Fix nodes un-draining after being drained due to unkillable step.
* Fix remote licenses allowed percentages reset to 0 during upgrade.
* sacct - Avoid truncating time strings when using SLURM_TIME_FORMAT with
the --parsable option.
* Fix regression in 22.05.0rc1 that broke Nodes=ALL in a NodeSet.
* openapi/v0.0.39 - fix jobs submitted via slurmrestd being allocated fewer
CPUs than tasks when requesting multiple tasks.
* Fix job not being scheduled on valid nodes and potentially being rejected
when using parentheses at the beginning of square brackets in a feature
request, for example: "feat1&[(feat2|feat3)]".
* Fix regression in 23.02.0rc1 which made --gres-flags=enforce-binding no
longer enforce optimal core-gpu job placement.
* mpi/pmix - Fix v5 to load correctly when libpmix.so isn't in the normal
lib path.
* data_parser/v0.0.39 - fix regression where "memory_per_node" would be
rejected for job submission.
* data_parser/v0.0.39 - fix regression where "memory_per_cpu" would be
rejected for job submission.
* slurmctld - add an assert to check for magic number presence before deleting
OBS-URL: https://build.opensuse.org/request/show/1076522
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=88
- updated to 23.02.1 with following chnages:
* job_container/tmpfs - cleanup job container even if namespace mount is
already unmounted.
* openapi/dbv0.0.38 - Fix not displaying an error when updating QOS or
associations fails.
* Fix nodes remaining as PLANNED after slurmctld save state recovery.
* Add cgroup.conf EnableControllers option for cgroup/v2.
* Get correct cgroup root to allow slurmd to run in containers like Docker.
* slurmctld - add missing PrivateData=jobs check to step ContainerID lookup
requests originated from 'scontrol show step container-id=<id>' or certain
scrun operations when container state can't be directly queried.
* Fix nodes un-draining after being drained due to unkillable step.
* Fix remote licenses allowed percentages reset to 0 during upgrade.
* sacct - Avoid truncating time strings when using SLURM_TIME_FORMAT with
the --parsable option.
* Fix regression in 22.05.0rc1 that broke Nodes=ALL in a NodeSet.
* openapi/v0.0.39 - fix jobs submitted via slurmrestd being allocated fewer
CPUs than tasks when requesting multiple tasks.
* Fix job not being scheduled on valid nodes and potentially being rejected
when using parentheses at the beginning of square brackets in a feature
request, for example: "feat1&[(feat2|feat3)]".
* Fix regression in 23.02.0rc1 which made --gres-flags=enforce-binding no
longer enforce optimal core-gpu job placement.
* mpi/pmix - Fix v5 to load correctly when libpmix.so isn't in the normal
lib path.
* data_parser/v0.0.39 - fix regression where "memory_per_node" would be
rejected for job submission.
* data_parser/v0.0.39 - fix regression where "memory_per_cpu" would be
rejected for job submission.
* slurmctld - add an assert to check for magic number presence before deleting
OBS-URL: https://build.opensuse.org/request/show/1076461
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=248
- updated to 23.02.0
* Highlights
+ slurmctld - Add new RPC rate limiting feature. This is enabled through
SlurmctldParameters=rl_enable, otherwise disabled by default.
+ Make scontrol reconfigure and sending a SIGHUP to the slurmctld behave
the same. If you were using SIGHUP as a 'lighter' scontrol reconfigure
to rotate logs please update your scripts to use SIGUSR2 instead.
+ Change cloud nodes to show by default. PrivateData=cloud is no longer
needed.
+ sreport - Count planned (FKA reserved) time for jobs running in
IGNORE_JOBS reservations. Previously was lumped into IDLE time.
+ job_container/tmpfs - Support running with an arbitrary list of private
mount points (/tmp and /dev/shm are the default, but not required).
+ job_container/tmpfs - Set more environment variables in InitScript.
+ Make all cgroup directories created by Slurm owned by root. This was the
behavior in cgroup/v2 but not in cgroup/v1 where by default the step
directories ownership were set to the user and group of the job.
+ accounting_storage/mysql - change purge/archive to calculate record ages
based on end time, rather than start or submission times.
+ job_submit/lua - add support for log_user() from slurm_job_modify().
+ Run the following scripts in slurmscriptd instead of slurmctld:
ResumeProgram, ResumeFailProgram, SuspendProgram, ResvProlog, ResvEpilog,
and RebootProgram (only with SlurmctldParameters=reboot_from_controller).
+ Only permit changing log levels with 'srun --slurmd-debug' by root
or SlurmUser.
+ slurmctld will fatal() when reconfiguring the job_submit plugin fails.
+ Add PowerDownOnIdle partition option to power down nodes after nodes
become idle.
+ Add "[jobid.stepid]" prefix from slurmstepd and "slurmscriptd" prefix
from slurmcriptd to Syslog logging. Previously was only happening when
OBS-URL: https://build.opensuse.org/request/show/1068320
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=83
+ Fixed GpuFreqDef option. When set in slurm.conf, it will be used if
--gpu-freq was not explicitly set by the job step.
+ topology/tree - Add new TopologyParam=SwitchAsNodeRank option to reorder
nodes based on switch layout. This can be useful if the naming convention
for the nodes does not natually map to the network topology.
+ Removed the default setting for GpuFreqDef. If unset, no attempt to change
the GPU frequency will be made if --gpu-freq is not set for the step.
OBS-URL: https://build.opensuse.org/request/show/1068316
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=232
- updated to 23.02.0-0rc1
* Highlights
+ slurmctld - Add new RPC rate limiting feature. This is enabled through
SlurmctldParameters=rl_enable, otherwise disabled by default.
+ Make scontrol reconfigure and sending a SIGHUP to the slurmctld behave
the same. If you were using SIGHUP as a 'lighter' scontrol reconfigure
to rotate logs please update your scripts to use SIGUSR2 instead.
+ Change cloud nodes to show by default. PrivateData=cloud is no longer
needed.
+ sreport - Count planned (FKA reserved) time for jobs running in
IGNORE_JOBS reservations. Previously was lumped into IDLE time.
+ job_container/tmpfs - Support running with an arbitrary list of private
mount points (/tmp and /dev/shm are the default, but not required).
+ job_container/tmpfs - Set more environment variables in InitScript.
+ Make all cgroup directories created by Slurm owned by root. This was the
behavior in cgroup/v2 but not in cgroup/v1 where by default the step
directories ownership were set to the user and group of the job.
+ accounting_storage/mysql - change purge/archive to calculate record ages
based on end time, rather than start or submission times.
+ job_submit/lua - add support for log_user() from slurm_job_modify().
+ Run the following scripts in slurmscriptd instead of slurmctld:
ResumeProgram, ResumeFailProgram, SuspendProgram, ResvProlog, ResvEpilog,
and RebootProgram (only with SlurmctldParameters=reboot_from_controller).
+ Only permit changing log levels with 'srun --slurmd-debug' by root
or SlurmUser.
+ slurmctld will fatal() when reconfiguring the job_submit plugin fails.
+ Add PowerDownOnIdle partition option to power down nodes after nodes
become idle.
+ Add "[jobid.stepid]" prefix from slurmstepd and "slurmscriptd" prefix
from slurmcriptd to Syslog logging. Previously was only happening when
OBS-URL: https://build.opensuse.org/request/show/1067475
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=231
- Test Suite fixes:
* Update README_Testsuite.md.
* Clean up left over files when de-installing test suite.
* Adjustment to test suite package: for SLE mark the openmpi4
devel package and slurm-hdf5 optional.
* Add -ffat-lto-objects to the build flags when LTO is set to
make sure the object files we ship with the test suite still
work correctly.
* Improve setup-testsuite.sh: copy ssh fingerprints from all nodes.
- set environment variable SUSE_ZNOW to 0 in %build to avoid module load
failures due to unresolved symbols as module take advantage of lazy
bindings (bsc#1200030).
OBS-URL: https://build.opensuse.org/request/show/1031255
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=80
* Update README_Testsuite.md.
* Clean up left over files when de-installing test suite.
* Adjustment to test suite package: for SLE mark the openmpi4
devel package and slurm-hdf5 optional.
* Add -ffat-lto-objects to the build flags when LTO is set to
make sure the object files we ship with the test suite still
work correctly.
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=224
- updated to 22.05.5
- NOTE: Slurm validates that libraries are of the same version. Unfortunately,
due to an oversight, we failed to notice that the slurmstepd loads the
hash_k12 library only after a job has completed. This means that if the
hash_k12 library is upgraded before a job finishes, the slurmstepd will load
the new library when the job finishes, and will fail due to a mismatch of
versions. This results in nodes with slurmstepd processes stuck
indefinitely. These processes require manual intervention to clean up. There
is no clean way to resolve these hung slurmstepd processes.
The only recommended way to upgrade between minor versions of 22.05 with
RPM’s or upgrades that replace current binaries and libraries is to drain the
nodes of running jobs first.
- Fixes a number of moderate severity issues, noteable are:
* Load hash plugin at slurmstepd launch time to prevent issues loading the
plugin at step completion if the Slurm installation is upgraded.
* Update nvml plugin to match the unique id format for MIG devices in new
Nvidia drivers.
* Fix multi-node step launch failure when nodes in the controller aren't in
natural order. This can happen with inconsistent node naming (such as
node15 and node052) or with dynamic nodes which can register in any order.
* job_container/tmpfs - cleanup containers even when the .ns file isn't
mounted anymore.
* Wait up to PrologEpilogTimeout before shutting down slurmd to allow prolog
and epilog scripts to complete or timeout. Previously, slurmd waited 120
seconds before timing out and killing prolog and epilog scripts. (forwarded request 1010642 from mslacken)
OBS-URL: https://build.opensuse.org/request/show/1030432
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=79
- updated to 22.05.5
- NOTE: Slurm validates that libraries are of the same version. Unfortunately,
due to an oversight, we failed to notice that the slurmstepd loads the
hash_k12 library only after a job has completed. This means that if the
hash_k12 library is upgraded before a job finishes, the slurmstepd will load
the new library when the job finishes, and will fail due to a mismatch of
versions. This results in nodes with slurmstepd processes stuck
indefinitely. These processes require manual intervention to clean up. There
is no clean way to resolve these hung slurmstepd processes.
The only recommended way to upgrade between minor versions of 22.05 with
RPM’s or upgrades that replace current binaries and libraries is to drain the
nodes of running jobs first.
- Fixes a number of moderate severity issues, noteable are:
* Load hash plugin at slurmstepd launch time to prevent issues loading the
plugin at step completion if the Slurm installation is upgraded.
* Update nvml plugin to match the unique id format for MIG devices in new
Nvidia drivers.
* Fix multi-node step launch failure when nodes in the controller aren't in
natural order. This can happen with inconsistent node naming (such as
node15 and node052) or with dynamic nodes which can register in any order.
* job_container/tmpfs - cleanup containers even when the .ns file isn't
mounted anymore.
* Wait up to PrologEpilogTimeout before shutting down slurmd to allow prolog
and epilog scripts to complete or timeout. Previously, slurmd waited 120
seconds before timing out and killing prolog and epilog scripts.
OBS-URL: https://build.opensuse.org/request/show/1010642
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=220
- Package the Slurm testsuite for QA purposes.
* Fixes for test suite:
Keep-logs-of-skipped-test-when-running-test-cases-sequentially.patch
Fix-test-21.41.patch
Fix-test-38.11.patch
Fix-test-32.8.patch
Fix-test-3.13.patch
Fix-test7.2-to-find-libpmix-under-lib64-as-well.patch
* Add documentation:
README_Testsuite.md
- Allow log in as user 'slurm'. This allows admins to run certain
priviledged commands more easily without becoming root. (forwarded request 988732 from eeich)
OBS-URL: https://build.opensuse.org/request/show/988733
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=74
- Package the Slurm testsuite for QA purposes.
* Fixes for test suite:
Keep-logs-of-skipped-test-when-running-test-cases-sequentially.patch
Fix-test-21.41.patch
Fix-test-38.11.patch
Fix-test-32.8.patch
Fix-test-3.13.patch
Fix-test7.2-to-find-libpmix-under-lib64-as-well.patch
* Add documentation:
README_Testsuite.md
- Allow log in as user 'slurm'. This allows admins to run certain
priviledged commands more easily without becoming root.
OBS-URL: https://build.opensuse.org/request/show/988732
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=207
NOTE: This package is not meant to be used for testing by the
user but rather for testing by the maintainers to ensure the
package is working properly.
DO NOT report test suite failures unless you are able to confirm
that the failure is really a bug.
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=205
- update to 22.05.0 with following changes:
- Support for dynamic node addition and removal
- Support for native Linux cgroup v2 operation
- Newly added plugins to support HPE Slingshot 11 networks
(switch/hpe_slingshot), and Intel Xe GPUs (gpu/oneapi)
- Added new acct_gather_interconnect/sysfs plugin to collect statistics
from arbitrary network interfaces.
- Expanded and synced set of environment variables available in the
Prolog/Epilog/PrologSlurmctld/EpilogSlurmctld scripts.
- New "--prefer" option to job submissions to allow for a "soft
constraint" request to influence node selection.
- Optional support for license planning in the backfill scheduler with
"bf_licenses" option in SchedulerParameters.
- removed file slurm-2.4.4-init.patch as sysvinit is now realy deprecated
- removed file load-pmix-major-version.patch as fixed upstream
OBS-URL: https://build.opensuse.org/request/show/980093
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=203
- Update to 21.08.7 with following changes:
* openapi/v0.0.37 - correct calculation for bf_queue_len_mean in /diag.
* Avoid shrinking a reservation when overlapping with downed nodes.
* Only check TRES limits against current usage for TRES requested by the job.
* Do not allocate shared gres (MPS) in whole-node allocations
* Constrain slurmstepd to job/step cgroup like in previous versions of Slurm.
* Fix warnings on 32-bit compilers related to printf() formats.
* Fix reconfigure issues after disabling/reenabling the GANG PreemptMode.
* Fix race condition where a cgroup was being deleted while another step
was creating it.
* Set the slurmd port correctly if multi-slurmd
* Fix FAIL mail not being sent if a job was cancelled due to preemption.
* slurmrestd - move debug logs for HTTP handling to be gated by debugflag
NETWORK to avoid unnecessary logging of communication contents.
* Fix issue with bad memory access when shrinking running steps.
* Fix various issues with internal job accounting with GRES when jobs are
shrunk.
* Fix ipmi polling on slurmd reconfig or restart.
* Fix srun crash when reserved ports are being used and het step fails
to launch.
* openapi/dbv0.0.37 - fix DELETE execution path on /user/{user_name}.
* slurmctld - Properly requeue all components of a het job if PrologSlurmctld
fails.
* rlimits - remove final calls to limit nofiles to 4096 but to instead use
the max possible nofiles in slurmd and slurmdbd.
* Allow the DBD agent to load large messages (up to MAX_BUF_SIZE) from state.
* Fix potential deadlock during slurmctld restart when there is a completing
job.
* slurmstepd - reduce user requested soft rlimits when they are above max
hard rlimits to avoid rlimit request being completely ignored and
OBS-URL: https://build.opensuse.org/request/show/974433
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=196
- update to 21.08.5 with following changes:
* Fix issue where typeless GRES node updates were not immediately reflected.
* Fix setting the default scrontab job working directory so that it's the home
of the different user (*u <user>) and not that of root or SlurmUser editor.
* Fix stepd not respecting SlurmdSyslogDebug.
* Fix concurrency issue with squeue.
* Fix job start time not being reset after launch when job is packed onto
already booting node.
* Fix updating SLURM_NODE_ALIASES for jobs packed onto powering up nodes.
* Cray - Fix issues with starting hetjobs.
* auth/jwks - Print fatal() message when jwks is configured but file could
not be opened.
* If sacctmgr has an association with an unknown qos as the default qos
print 'UNKN*###' instead of leaving a blank name.
* Correctly determine task count when giving --cpus-per-gpu, --gpus and
*-ntasks-per-node without task count.
* slurmctld - Fix places where the global last_job_update was not being set
to the time of update when a job's reason and description were updated.
* slurmctld - Fix case where a job submitted with more than one partition
would not have its reason updated while waiting to start.
* Fix memory leak in node feature rebooting.
* Fix time limit permanetly set to 1 minute by backfill for job array tasks
higher than the first with QOS NoReserve flag and PreemptMode configured.
* Fix sacct -N to show jobs that started in the current second
* Fix issue on running steps where both SLURM_NTASKS_PER_TRES and
SLURM_NTASKS_PER_GPU are set.
* Handle oversubscription request correctly when also requesting
*-ntasks-per-tres.
* Correctly detect when a step requests bad gres inside an allocation.
* slurmstepd - Correct possible deadlock when UnkillableStepTimeout triggers.
OBS-URL: https://build.opensuse.org/request/show/942081
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=195
in 21.08 tree.
* CVE-2021-43337:
For sites using the new AccountingStoreFlags=job_script and/or job_env
options, an issue was reported with the access control rules in SlurmDBD
that will permit users to request job scripts and environment files that
they should not have access to. (Scripts/environments are meant to only be
accessible by user accounts with administrator privileges, by account
coordinators for jobs submitted under their account, and by the user
themselves.)
- changes from 21.08.3:
* This includes a number of fixes since the last release a month ago,
including one critical fix to prevent a communication issue between
slurmctld and slurmdbd for sites that have started using the new
AccountingStoreFlags=job_script functionality.
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=193
- update to 21.08.2
- major change:
* removed of support of the TaskAffinity=yes option in cgroup.conf. Please
consider using "TaskPlugins=cgroup,affinity" in slurm.conf as an option.
- minor changes and bugfixes:
* slurmctld - fix how the max number of cores on a node in a partition are
calculated when the partition contains multi*socket nodes. This in turn
corrects certain jobs node count estimations displayed client*side.
* job_submit/cray_aries - fix "craynetwork" GRES specification after changes
introduced in 21.08.0rc1 that made TRES always have a type prefix.
* Ignore nonsensical check in the slurmd for [Pro|Epi]logSlurmctld.
* Fix writing to stderr/syslog when systemd runs slurmctld in the foreground.
* Fix issue with updating job started with node range.
* Fix issue with nodes not clearing state in the database when the slurmctld
is started with clean*start.
* Fix hetjob components > 1 timing out due to InactiveLimit.
* Fix sprio printing -nan for normalized association priority if
PriorityWeightAssoc was not defined.
* Disallow FirstJobId=0.
* Preserve job start info in the database for a requeued job that hadn't
registered the first time in the database yet.
* Only send one message on prolog failure from the slurmd.
* Remove support for TaskAffinity=yes in cgroup.conf.
* accounting_storage/mysql - fix issue where querying jobs via sacct
*-whole-hetjob=yes or slurmrestd (which automatically includes this flag)
could in some cases return more records than expected.
* Fix issue for preemption of job array task that makes afterok dependency
fail. Additionally, send emails when requeueing happens due to preemption.
* Fix sending requeue mail type.
* Properly resize a job's GRES bitmaps and counts when resizing the job.
OBS-URL: https://build.opensuse.org/request/show/923425
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=190
- updated to 21.08.1 with following bug fixes:
* Fix potential memory leak if a problem happens while allocating GRES for
a job.
* If an overallocation of GRES happens terminate the creation of a job.
* AutoDetect=nvml: Fatal if no devices found in MIG mode.
* Print federation and cluster sacctmgr error messages to stderr.
* Fix off by one error in --gpu-bind=mask_gpu.
* Add --gpu-bind=none to disable gpu binding when using --gpus-per-task.
* Handle the burst buffer state "alloc-revoke" which previously would not
display in the job correctly.
* Fix issue in the slurmstepd SPANK prolog/epilog handler where configuration
values were used before being initialized.
* Restore a step's ability to utilize all of an allocations memory if --mem=0.
* Fix --cpu-bind=verbose garbage taskid.
* Fix cgroup task affinity issues from garbage taskid info.
* Make gres_job_state_validate() client logging behavior as before 44466a4641.
* Fix steps with --hint overriding an allocation with --threads-per-core.
* Require requesting a GPU if --mem-per-gpu is requested.
* Return error early if a job is requesting --ntasks-per-gpu and no gpus or
task count.
* Properly clear out pending step if unavailable to run with available
resources.
* Kill all processes spawned by burst_buffer.lua including decendents.
* openapi/v0.0.{35,36,37} - Avoid setting default values of min_cpus,
job name, cwd, mail_type, and contiguous on job update.
* openapi/v0.0.{35,36,37} - Clear user hold on job update if hold=false.
* Prevent CRON_JOB flag from being cleared when loading job state.
* sacctmgr - Fix deleting WCKeys when not specifying a cluster.
* Fix getting memory for a step when the first node in the step isn't the
first node in the allocation.
OBS-URL: https://build.opensuse.org/request/show/919668
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=186
- updated to 21.08.1, major changes:
* A new "AccountingStoreFlags=job_script" option to store the job scripts
directly in SlurmDBD.
* Added "sacct -o SubmitLine" format option to get the submit line
of a job/step.
* Changes to the node state management so that nodes are marked as PLANNED
instead of IDLE if the scheduler is still accumulating resources while
waiting to launch a job on them.
* RS256 token support in auth/jwt.
* Overhaul of the cgroup subsystems to simplify operation, mitigate a number
of inherent race conditions, and prepare for future cgroup v2 support.
* Further improvements to cloud node power state management.
* A new child process of the Slurm controller called "slurmscriptd"
responsible for executing PrologSlurmctld and EpilogSlurmctld scripts,
which significantly reduces performance issues associated with enabling
those options.
* A new burst_buffer/lua plugin allowing for site-specific asynchronous job
data management.
* Fixes to the job_container/tmpfs plugin to allow the slurmd process to be
restarted while the job is running without issue.
* Added json/yaml output to sacct, squeue, and sinfo commands.
* Added a new node_features/helpers plugin to provide a generic way to change
settings on a compute node across a reboot.
* Added support for automatically detecting and broadcasting shared libraries
for an executable launched with "srun --bcast".
* Added initial OCI container execution support with a new --container option
to sbatch and srun.
* Improved "configless" support by allowing multiple control servers to be
specified through the slurmd --conf-server option, and send additional
configuration files at startup including cli_filter.lua.
OBS-URL: https://build.opensuse.org/request/show/915777
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=184
- Updated to 20.11.8:
* slurmctld - fix erroneous "StepId=CORRUPT" messages in error logs.
* Correct the error given when auth plugin fails to pack a credential.
* Fix unused-variable compiler warning on FreeBSD in fd_resolve_path().
* acct_gather_filesystem/lustre - only emit collection error once per step.
* Add GRES environment variables (e.g., CUDA_VISIBLE_DEVICES) into the
interactive step, the same as is done for the batch step.
* Fix various potential deadlocks when altering objects in the database
dealing with every cluster in the database.
* slurmrestd:
- handle slurmdbd connection failures without segfaulting.
- fix segfault for searches in slurmdb/v0.0.36/jobs.
- remove (non-functioning) users query parameter for
slurmdb/v0.0.36/jobs from openapi.json
- fix segfault in slurmrestd db/jobs with numeric queries
- add argv handling for job/submit endpoint.
- add description for slurmdb/job endpoint.
* slurmrestd/dbv0.0.36:
- Fix values dumped in job state/current and
job step state.
- Correct description for previous state property.
* srun:
- fix broken node step allocation in a heterogeneous allocation.
- leave SLURM_DIST_UNKNOWN as default for --interactive.
* Fail step creation if -n is not multiple of --ntasks-per-gpu.
* job_container/tmpfs - Fix slowdown on teardown.
* Fix problem with SlurmctldProlog where requeued jobs would never launch.
* job_container/tmpfs - Fix issue when restarting slurmd where the namespace
mount points could disappear.
* sacct:
OBS-URL: https://build.opensuse.org/request/show/903744
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=183
- Updated to 20.11.7 which fixes CVE-2021-31215 (bsc#1186024)
- New featuresi from 20.11.7:
* slurmd - handle configless failures gracefully instead of hanging
indefinitely.
* select/cons_tres - fix Dragonfly topology not selecting nodes in the same
leaf switch when it should as well as requests with *-switches option.
* Fix issue where certain step requests wouldn't run if the first node in the
job allocation was full and there were idle resources on other nodes in
the job allocation.
* Fix deadlock issue with <Prolog|Epilog>Slurmctld.
* torque/qstat - fix printf error message in output.
* When adding associations or wckeys avoid checking multiple times a user or
cluster name.
* Fix wrong jobacctgather information on a step on multiple nodes
due to timeouts sending its the information gathered on its node.
* Fix missing xstrdup which could result in slurmctld segfault on array jobs.
* Fix security issue in PrologSlurmctld and EpilogSlurmctld by always
prepending SPANK_ to all user-set environment variables. CVE-2021-31215.
- New features from 20.11.6:
* Fix sacct assert with the --qos option.
* Use pkg-config --atleast-version instead of --modversion for systemd.
* common/fd - fix getsockopt() call in fd_get_socket_error().
* Properly handle the return from fd_get_socket_error() in _conn_readable().
* cons_res - Fix issue where running jobs were not taken into consideration
when creating a reservation.
* Avoid a deadlock between job_list for_each and assoc QOS_LOCK.
* Fix TRESRunMins usage for partition qos on restart/reconfig.
* Fix printing of number of tasks on a completed job that didn't request
tasks.
* Fix updating GrpTRESRunMins when decrementing job time is bigger than it.
OBS-URL: https://build.opensuse.org/request/show/893087
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=59
- New featuresi from 20.11.7:
* slurmd - handle configless failures gracefully instead of hanging
indefinitely.
* select/cons_tres - fix Dragonfly topology not selecting nodes in the same
leaf switch when it should as well as requests with *-switches option.
* Fix issue where certain step requests wouldn't run if the first node in the
job allocation was full and there were idle resources on other nodes in
the job allocation.
* Fix deadlock issue with <Prolog|Epilog>Slurmctld.
* torque/qstat - fix printf error message in output.
* When adding associations or wckeys avoid checking multiple times a user or
cluster name.
* Fix wrong jobacctgather information on a step on multiple nodes
due to timeouts sending its the information gathered on its node.
* Fix missing xstrdup which could result in slurmctld segfault on array jobs.
* Fix security issue in PrologSlurmctld and EpilogSlurmctld by always
prepending SPANK_ to all user-set environment variables. CVE-2021-31215.
- New features from 20.11.6:
* Fix sacct assert with the --qos option.
* Use pkg-config --atleast-version instead of --modversion for systemd.
* common/fd - fix getsockopt() call in fd_get_socket_error().
* Properly handle the return from fd_get_socket_error() in _conn_readable().
* cons_res - Fix issue where running jobs were not taken into consideration
when creating a reservation.
* Avoid a deadlock between job_list for_each and assoc QOS_LOCK.
* Fix TRESRunMins usage for partition qos on restart/reconfig.
* Fix printing of number of tasks on a completed job that didn't request
tasks.
* Fix updating GrpTRESRunMins when decrementing job time is bigger than it.
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=179
- Udpate to 20.11.04
* Fix node selection for advanced reservations with features.
* mpi/pmix: Handle pipe failure better when using ucx.
* mpi/pmix: include PMIX_NODEID for each process entry.
* Fix job getting rejected after being requeued on same node that died.
* job_submit/lua - add "network" field.
* Fix situations when a reoccuring reservation could erroneously skip a
period.
* Ensure that a reservations [pro|epi]log are ran on reoccuring reservations.
* Fix threads-per-core memory allocation issue when using CR_CPU_MEMORY.
* Fix scheduling issue with --gpus.
* Fix gpu allocations that request --cpus-per-task.
* mpi/pmix: fixed print messages for all PMIXP_* macros
* Add mapping for XCPU to --signal option.
* Fix regression in 20.11 that prevented a full pass of the main scheduler
from ever executing.
* Work around a glibc bug in which "0" is incorrectly printed as "nan"
which will result in corrupted association state on restart.
* Fix regression in 20.11 which made slurmd incorrectly attempt to find the
parent slurmd address when not applicable and send incorrect reverse*tree
info to the slurmstepd.
* Fix cgroup ns detection when using containers (e.g. LXC or Docker).
* scrontab - change temporary file handling to work with emacs.
- Removed check-for-lipmix.so.MAJOR.patch
- Added: load-pmix-major-version.patch
OBS-URL: https://build.opensuse.org/request/show/874647
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=173
- Update to 20.11.03
- This release includes a major functional change to how job step launch is
handled compared to the previous 20.11 releases. This affects srun as
well as MPI stacks - such as Open MPI - which may use srun internally as
part of the process launch.
One of the changes made in the Slurm 20.11 release was to the semantics
for job steps launched through the 'srun' command. This also
inadvertently impacts many MPI releases that use srun underneath their
own mpiexec/mpirun command.
For 20.11.{0,1,2} releases, the default behavior for srun was changed
such that each step was allocated exactly what was requested by the
options given to srun, and did not have access to all resources assigned
to the job on the node by default. This change was equivalent to Slurm
setting the --exclusive option by default on all job steps. Job steps
desiring all resources on the node needed to explicitly request them
through the new '--whole' option.
In the 20.11.3 release, we have reverted to the 20.02 and older behavior
of assigning all resources on a node to the job step by default.
This reversion is a major behavioral change which we would not generally
do on a maintenance release, but is being done in the interest of
restoring compatibility with the large number of existing Open MPI (and
other MPI flavors) and job scripts that exist in production, and to
remove what has proven to be a significant hurdle in moving to the new
release.
Please note that one change to step launch remains - by default, in
20.11 steps are no longer permitted to overlap on the resources they
have been assigned. If that behavior is desired, all steps must
explicitly opt-in through the newly added '--overlap' option.
Further details and a full explanation of the issue can be found at:
https://bugs.schedmd.com/show_bug.cgi?id=10383#c63
OBS-URL: https://build.opensuse.org/request/show/864993
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=171
- Add support for configuration files from external plugins.
While built-in plugins have their configuration added in slurm.conf,
external SPANK plugins add their configuration to plugstack.conf
To allow packaging easily spank plugins, their configuration files
should be added independently at /etc/spack/plugstack.conf.d and
plugstack.conf should be left with an oneliner including all the
files under /etc/spack/plugstack.conf.d
OBS-URL: https://build.opensuse.org/request/show/860690
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=164
- Update to 20.11.02
* Fix older versions of sacct not working with 20.11.
* Fix slurmctld crash when using a pre-20.11 srun in a job allocation.
* Correct logic problem in _validate_user_access.
* Fix libpmi to initialize Slurm configuration correctly.
- Update to 20.11.01
* Fix spelling of "overcomited" to "overcomitted" in sreport's cluster
utilization report.
* Silence debug message about shutting down backup controllers if none are
configured.
* Don't create interactive srun until PrologSlurmctld is done.
* Fix fd symlink path resolution.
* Fix slurmctld segfault on subnode reservation restore after node
configuration change.
* Fix resource allocation response message environment allocation size.
* Ensure that details->env_sup is NULL terminated.
* select/cray_aries - Correctly remove jobs/steps from blades using NPC.
* cons_tres - Avoid max_node_gres when entire node is allocated with
--ntasks-per-gpu.
* Allow NULL arg to data_get_type().
* In sreport have usage for a reservation contain all jobs that ran in the
reservation instead of just the ones that ran in the time specified. This
matches the report for the reservation is not truncated for a time period.
* Fix issue with sending wrong batch step id to a < 20.11 slurmd.
* Add a job's alloc_node to lua for job modification and completion.
* Fix regression getting a slurmdbd connection through the perl API.
* Stop the extern step terminate monitor right after proctrack_g_wait().
* Fix removing the normalized priority of assocs.
* slurmrestd/v0.0.36 - Use correct name for partition field:
"min nodes per job" -"min_nodes_per_job".
OBS-URL: https://build.opensuse.org/request/show/859114
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=162
- Update to version 20.11.0
Slurm 20.11 includes a number of new features including:
* Overhaul of the job step management and launch code, alongside improved
GPU task placement support.
* A new "Interactive Step" mode of operation for salloc.
* A new "scrontab" command that can be used to submit and manage
periodically repeating jobs.
* IPv6 support.
* Changes to the reservation logic, with new options allowing users
to delete reservations, allowing admins to skip the next occurance of a
repeated reservation, and allowing for a job to be submitted and eligible
to run within multiple reservations.
* Dynamic Future Nodes - automatically associate a dynamically
provisioned (or "cloud") node against a NodeName definition with matching
hardware.
* An experimental new RPC queuing mode for slurmctld to reduce thread
contention on heavily loaded clusters.
* SlurmDBD integration with the Slurm REST API.
Also check
https://github.com/SchedMD/slurm/blob/slurm-20-11-0-1/RELEASE_NOTES (forwarded request 852039 from eeich)
OBS-URL: https://build.opensuse.org/request/show/853268
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=50
- Update to version 20.11.0
Slurm 20.11 includes a number of new features including:
* Overhaul of the job step management and launch code, alongside improved
GPU task placement support.
* A new "Interactive Step" mode of operation for salloc.
* A new "scrontab" command that can be used to submit and manage
periodically repeating jobs.
* IPv6 support.
* Changes to the reservation logic, with new options allowing users
to delete reservations, allowing admins to skip the next occurance of a
repeated reservation, and allowing for a job to be submitted and eligible
to run within multiple reservations.
* Dynamic Future Nodes - automatically associate a dynamically
provisioned (or "cloud") node against a NodeName definition with matching
hardware.
* An experimental new RPC queuing mode for slurmctld to reduce thread
contention on heavily loaded clusters.
* SlurmDBD integration with the Slurm REST API.
Also check
https://github.com/SchedMD/slurm/blob/slurm-20-11-0-1/RELEASE_NOTES
OBS-URL: https://build.opensuse.org/request/show/852039
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=160
- Updated to 20.02.5, changes:
* Fix leak of TRESRunMins when job time is changed with --time-min
* pam_slurm - explicitly initialize slurm config to support configless mode.
* scontrol - Fix exit code when creating/updating reservations with wrong
Flags.
* When a GRES has a no_consume flag, report 0 for allocated.
* Fix cgroup cleanup by jobacct_gather/cgroup.
* When creating reservations/jobs don't allow counts on a feature unless
using an XOR.
* Improve number of boards discovery
* Fix updating a reservation NodeCnt on a zero-count reservation.
* slurmrestd - provide an explicit error messages when PSK auth fails.
* cons_tres - fix job requesting single gres per-node getting two or more
nodes with less CPUs than requested per-task.
* cons_tres - fix calculation of cores when using gres and cpus-per-task.
* cons_tres - fix job not getting access to socket without GPU or with less
than --gpus-per-socket when not enough cpus available on required socket
and not using --gres-flags=enforce binding.
* Fix HDF5 type version build error.
* Fix creation of CoreCnt only reservations when the first node isn't
available.
* Fix wrong DBD Agent queue size in sdiag when using accounting_storage/none.
* Improve job constraints XOR option logic.
* Fix preemption of hetjobs when needed nodes not in leader component.
* Fix wrong bit_or() messing potential preemptor jobs node bitmap, causing
bad node deallocations and even allocation of nodes from other partitions.
* Fix double-deallocation of preempted non-leader hetjob components.
* slurmdbd - prevent truncation of the step nodelists over 4095.
* Fix nodes remaining in drain state state after rebooting with ASAP option.
- changes from 20.02.4:
OBS-URL: https://build.opensuse.org/request/show/845108
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=156
- Updated to 20.02.3 which fixes CVE-2020-12693 (bsc#1172004).
- Other changes are:
* Factor in ntasks-per-core=1 with cons_tres.
* Fix formatting in error message in cons_tres.
* Fix calling stat on a NULL variable.
* Fix minor memory leak when using reservations with flags=first_cores.
* Fix gpu bind issue when CPUs=Cores and ThreadsPerCore > 1 on a node.
* Fix --mem-per-gpu for heterogenous --gres requests.
* Fix slurmctld load order in load_all_part_state().
* Fix race condition not finding jobacct gather task cgroup entry.
* Suppress error message when selecting nodes on disjoint topologies.
* Improve performance of _pack_default_job_details() with large number of job
* arguments.
* Fix archive loading previous to 17.11 jobs per-node req_mem.
* Fix regresion validating that --gpus-per-socket requires --sockets-per-node
* for steps. Should only validate allocation requests.
* error() instead of fatal() when parsing an invalid hostlist.
* nss_slurm - fix potential deadlock in slurmstepd on overloaded systems.
* cons_tres - fix --gres-flags=enforce-binding and related --cpus-per-gres.
* cons_tres - Allocate lowest numbered cores when filtering cores with gres.
* Fix getting system counts for named GRES/TRES.
* MySQL - Fix for handing typed GRES for association rollups.
* Fix step allocations when tasks_per_core > 1.
* Fix allocating more GRES than requested when asking for multiple GRES types.
OBS-URL: https://build.opensuse.org/request/show/808569
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=45
- Updated to 20.02.3 which fixes CVE-2020-12693
- Other changes are:
* Factor in ntasks-per-core=1 with cons_tres.
* Fix formatting in error message in cons_tres.
* Fix calling stat on a NULL variable.
* Fix minor memory leak when using reservations with flags=first_cores.
* Fix gpu bind issue when CPUs=Cores and ThreadsPerCore > 1 on a node.
* Fix --mem-per-gpu for heterogenous --gres requests.
* Fix slurmctld load order in load_all_part_state().
* Fix race condition not finding jobacct gather task cgroup entry.
* Suppress error message when selecting nodes on disjoint topologies.
* Improve performance of _pack_default_job_details() with large number of job
* arguments.
* Fix archive loading previous to 17.11 jobs per-node req_mem.
* Fix regresion validating that --gpus-per-socket requires --sockets-per-node
* for steps. Should only validate allocation requests.
* error() instead of fatal() when parsing an invalid hostlist.
* nss_slurm - fix potential deadlock in slurmstepd on overloaded systems.
* cons_tres - fix --gres-flags=enforce-binding and related --cpus-per-gres.
* cons_tres - Allocate lowest numbered cores when filtering cores with gres.
* Fix getting system counts for named GRES/TRES.
* MySQL - Fix for handing typed GRES for association rollups.
* Fix step allocations when tasks_per_core > 1.
* Fix allocating more GRES than requested when asking for multiple GRES types.
OBS-URL: https://build.opensuse.org/request/show/808130
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=147
- Updated to 20.02.1 with following changes"
* Improve job state reason for jobs hitting partition_job_depth.
* Speed up testing of singleton dependencies.
* Fix negative loop bound in cons_tres.
* srun - capture the MPI plugin return code from mpi_hook_client_fini() and
use as final return code for step failure.
* Fix segfault in cli_filter/lua.
* Fix --gpu-bind=map_gpu reusability if tasks > elements.
* Make sure config_flags on a gres are sent to the slurmctld on node
registration.
* Prolog/Epilog - Fix missing GPU information.
* Fix segfault when using config parser for expanded lines.
* Fix bit overlap test function.
* Don't accrue time if job begin time is in the future.
* Remove accrue time when updating a job start/eligible time to the future.
* Fix regression in 20.02.0 that broke --depend=expand.
* Reset begin time on job release if it's not in the future.
* Fix for recovering burst buffers when using high-availability.
* Fix invalid read due to freeing an incorrectly allocated env array.
* Update slurmctld -i message to warn about losing data.
* Fix scontrol cancel_reboot so it clears the DRAIN flag and node reason for a
pending ASAP reboot.
OBS-URL: https://build.opensuse.org/request/show/788905
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=145
- Update to version 20.02.0 (jsc#SLE-8491)
* Fix minor memory leak in slurmd on reconfig.
* Fix invalid ptr reference when rolling up data in the database.
* Change shtml2html.py to require python3 for RHEL8 support, and match
man2html.py.
* slurm.spec - override "hardening" linker flags to ensure RHEL8 builds
in a usable manner.
* Fix type mismatches in the perl API.
* Prevent use of uninitialized slurmctld_diag_stats.
* Fixed various Coverity issues.
* Only show warning about root-less topology in daemons.
* Fix accounting of jobs in IGNORE_JOBS reservations.
* Fix issue with batch steps state not loading correctly when upgrading from
19.05.
* Deprecate max_depend_depth in SchedulerParameters and move it to
DependencyParameters.
* Silence erroneous error on slurmctld upgrade when loading federation state.
* Break infinite loop in cons_tres dealing with incorrect tasks per tres
request resulting in slurmctld hang.
* Improve handling of --gpus-per-task to make sure appropriate number of GPUs
is assigned to job.
* Fix seg fault on cons_res when requesting --spread-job.
- Move to python3 for everything but SLE-11-SP4
* For SLE-11-SP4 add a workaround to handle a python3 script (python2.7
compliant).
OBS-URL: https://build.opensuse.org/request/show/779379
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=136
- Update to version 20.02.0-rc1
* sbatch - fix segfault when no newline at the end of a burst buffer file.
* Change scancel to only check job's base state when matching -t options.
* Save job dependency list in state files.
* cons_tres - allow jobs to be run on systems with root-less topologies.
* Restore pre-20.02pre1 PrologSlurmctld synchonization behavior to avoid
various race conditions, and ensure proper batch job launch.
* Add new slurmrestd command/daemon which implements the Slurm REST API.
- Update to version 20.02.0-0pre1, highlights are
OBS-URL: https://build.opensuse.org/request/show/774250
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=134
- Updated to version 20.02.0-0pre1, highlights are
Highlights:
* Exclusive behavior of a node includes all GRES on a node as well
as the cpus.
* Use python3 instead of python for internal build/test scripts.
The slurm.spec file has been updated to depend on python3 as well.
* Added new NodeSet configuration option to help simplify partition
configuration sections for heterogeneous / condo*style clusters.
* Added slurm.conf option MaxDBDMsgs to control how many messages will be
stored in the slurmctld before throwing them away when the slurmdbd is down.
* The checkpoint plugin interface and all associated API calls have been
removed.
* slurm_init_job_desc_msg() initializes mail_type as uint16_t. This allows
mail_type to be set to NONE with scontrol.
* Add new slurm_spank_log() function to print messages back to the user from
within a SPANK plugin without prepending "error: " from slurm_error().
* Enforce having partition name and nodelist=ALL when creating reservations
with flags=PART_NODES.
* SPANK - removed never-implemented slurm_spank_slurmd_init() interface. This
hook has always been accessible through slurm_spank_init() in the
S_CTX_SLURMD context instead.
* sbcast - add new BcastAddr option to NodeName lines to allow sbcast traffic
to flow over an alternate network path.
* Added auth/jwt plugin, and 'scontrol token' subcommand. PMIx - improve
* performance of proc map generation. Deprecate kill_invalid_depend in
* SchedulerParameters and move it to a new
option called DependencyParameters.
* Enable job dependencies for any job on any cluster in the same federation.
* Allow clusters to be added automatically to db at startup of ctld. Add
* AccountingStorageExternalHost slurm.conf parameter. The
OBS-URL: https://build.opensuse.org/request/show/773459
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=130
- BuildRequire pkgconfig(systemd) instead of systemd: allow OBS to
shortcut through the -mini flavors.
- Use systemd_ordering instead of systemd_requires: systemd is
never a strict requirement; but in case the system is scheduled
for installation together with systemd, we want systemd to be
installed prior to slurm.
- start slurmdbd after mariadb (bsc#1161716)
OBS-URL: https://build.opensuse.org/request/show/766872
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=123
- Update to version 19.05.5 (jsc#SLE-8491)
* Check %docdir/NEWS for details.
* Includes security fixes CVE-2019-19727, CVE-2019-19728,
CVE-2019-12838.
* Disable i586 builds as this is no longer supported.
* Create libnss_slurm package to support user and group resolution
thru slurmstepd.
* slurm-2.4.4-rpath.patch -> Remove-rpath-from-build.patch
Obsoleted:
- pam_slurm_adopt-avoid-running-outside-of-the-sshd-PA.patch
- pam_slurm_adopt-send_user_msg-don-t-copy-undefined-d.patch
- pam_slurm_adopt-use-uid-to-determine-whether-root-is.patch
OBS-URL: https://build.opensuse.org/request/show/762650
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=118
- Deprecate "ControlMachine" only for SLURM version upgrades and
products newer than 1501. This ensures that the original setting
is retained for the SLURM version shipped origianlly with SLE-15-SP1
or Leap 15.1.
- Update to v18.08.9 for fixing CVE-2019-19728 (bsc#1159692).
* Wrap END_TIMER{,2,3} macro definition in "do {} while (0)" block.
* Make sview work with glib2 v2.62.
* Make Slurm compile on linux after sys/sysctl.h was deprecated.
* Install slurmdbd.conf.example with 0600 permissions to encourage secure
use. CVE-2019-19727.
* srun - do not continue with job launch if --uid fails. CVE-2019-19728.
- added pmix support jsc#SLE-10800
- Use --with-shared-libslurm to build slurm binaries using libslurm.
- Make libslurm depend on slurm-config.
- Fix ownership of /var/spool/slurm on new installations
and upgrade (boo#1158696).
- Fix permissions of slurmdbd.conf (bsc#1155784, CVE-2019-19727).
- Fix %posttrans macro _res_update to cope with added newline
(bsc#1153259).
- Add package slurm-webdoc which sets up a web server to provide
the documentation for the version shipped.
- Move srun from 'slurm' to 'slurm-node': srun is required on the
nodes as well so sbatch will work. 'slurm-node' is a requirement (forwarded request 760450 from eeich)
OBS-URL: https://build.opensuse.org/request/show/761961
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=34
- Deprecate "ControlMachine" only for SLURM version upgrades and
products newer than 1501. This ensures that the original setting
is retained for the SLURM version shipped origianlly with SLE-15-SP1
or Leap 15.1.
- Update to v18.08.9 for fixing CVE-2019-19728 (bsc#1159692).
* Wrap END_TIMER{,2,3} macro definition in "do {} while (0)" block.
* Make sview work with glib2 v2.62.
* Make Slurm compile on linux after sys/sysctl.h was deprecated.
* Install slurmdbd.conf.example with 0600 permissions to encourage secure
use. CVE-2019-19727.
* srun - do not continue with job launch if --uid fails. CVE-2019-19728.
- added pmix support jsc#SLE-10800
- Use --with-shared-libslurm to build slurm binaries using libslurm.
- Make libslurm depend on slurm-config.
- Fix ownership of /var/spool/slurm on new installations
and upgrade (boo#1158696).
- Fix permissions of slurmdbd.conf (bsc#1155784, CVE-2019-19727).
- Fix %posttrans macro _res_update to cope with added newline
(bsc#1153259).
- Add package slurm-webdoc which sets up a web server to provide
the documentation for the version shipped.
- Move srun from 'slurm' to 'slurm-node': srun is required on the
nodes as well so sbatch will work. 'slurm-node' is a requirement
OBS-URL: https://build.opensuse.org/request/show/760450
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=116
- added cray depend libraries to seperate package, as they are now
built, since json is enabled
- Updated to 18.0.7 for fixing CVE-2019-12838 and (bsc#1140709)
* Update "xauth list" to use the same 10000ms timeout as the other xauth
commands.
* Fix issue in gres code to handle a gres cnt of 0.
* Don't purge jobs if backfill is running.
* Verify job is pending add/removing accrual time.
* Don't abort when the job doesn't have an association that was removed
before the job was able to make it to the database.
* Set state_reason if select_nodes() fails job for QOS or Account.
* Avoid seg_fault on referencing association without a valid_qos bitmap.
* If Association/QOS is removed on a pending job set that job as ineligible.
* When changing a jobs account/qos always make sure you remove the old limits.
* Don't reset a FAIL_QOS or FAIL_ACCOUNT job reason until the qos or
account changed.
* Restore "sreport -T ALL" functionality.
* Correctly typecast signals being sent through the api.
* Properly initialize structures throughout Slurm.
* Sync "numtask" squeue format option for jobs and steps to "numtasks".
* Fix sacct -PD to avoid CA before start jobs.
* Fix potential deadlock with backup slurmctld.
* Fixed issue with jobs not appearing in sacct after dependency satisfied.
* Fix showing non-eligible jobs when asking with -j and not -s.
* Fix issue with backfill scheduler scheduling tasks of an array
when not the head job.
* accounting_storage/mysql - fix SIGABRT in the archive load logic.
* accounting_storage/mysql - fix memory leak in the archive load logic.
* Limit records per single SQL statement when loading archived data. (forwarded request 714908 from mslacken)
OBS-URL: https://build.opensuse.org/request/show/714909
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=28
- added cray depend libraries to seperate package, as they are now
built, since json is enabled
- Updated to 18.0.7 for fixing CVE-2019-12838 and (bsc#1140709)
* Update "xauth list" to use the same 10000ms timeout as the other xauth
commands.
* Fix issue in gres code to handle a gres cnt of 0.
* Don't purge jobs if backfill is running.
* Verify job is pending add/removing accrual time.
* Don't abort when the job doesn't have an association that was removed
before the job was able to make it to the database.
* Set state_reason if select_nodes() fails job for QOS or Account.
* Avoid seg_fault on referencing association without a valid_qos bitmap.
* If Association/QOS is removed on a pending job set that job as ineligible.
* When changing a jobs account/qos always make sure you remove the old limits.
* Don't reset a FAIL_QOS or FAIL_ACCOUNT job reason until the qos or
account changed.
* Restore "sreport -T ALL" functionality.
* Correctly typecast signals being sent through the api.
* Properly initialize structures throughout Slurm.
* Sync "numtask" squeue format option for jobs and steps to "numtasks".
* Fix sacct -PD to avoid CA before start jobs.
* Fix potential deadlock with backup slurmctld.
* Fixed issue with jobs not appearing in sacct after dependency satisfied.
* Fix showing non-eligible jobs when asking with -j and not -s.
* Fix issue with backfill scheduler scheduling tasks of an array
when not the head job.
* accounting_storage/mysql - fix SIGABRT in the archive load logic.
* accounting_storage/mysql - fix memory leak in the archive load logic.
* Limit records per single SQL statement when loading archived data.
OBS-URL: https://build.opensuse.org/request/show/714908
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=100
- Update SLURM to 18.08.7:
* Set debug statement to debug2 to avoid benign error messages.
* Add SchedulerParameters option of bf_hetjob_immediate to attempt to start
a heterogeneous job as soon as all of its components are determined able
to do so.
* Fix underflow causing decay thread to exit.
* Fix main scheduler not considering hetjobs when building the job queue.
* Fix regression for sacct to display old jobs without a start time.
* Fix setting correct number of gres topology bits.
* Update hetjobs pending state reason when appropriate.
* Fix accounting_storage/filetxt's understanding of TRES.
* Set Accrue time when not enforcing limits.
* Fix srun segfault when requesting a hetjob with test_exec or bcast
options.
* Hide multipart priorities log message behind Priority debug flag.
* sched/backfill - Make hetjobs sensitive to bf_max_job_start.
* Fix slurmctld segfault due to job's partition pointer NULL dereference.
* Fix issue with OR'ed job dependencies.
* Add new job's bit_flags of INVALID_DEPEND to prevent rebuilding a job's
dependency string when it has at least one invalid and purged dependency.
* Promote federation unsynced siblings log message from debug to info.
* burst_buffer/cray - fix slurmctld SIGABRT due to illegal read/writes.
* burst_buffer/cray - fix memory leak due to unfreed job script content.
* node_features/knl_cray - fix script_argv use-after-free.
* burst_buffer/cray - fix script_argv use-after-free.
* Fix invalid reads of size 1 due to non null-terminated string reads.
* Add extra debug2 logs to identify why BadConstraints reason is set.
OBS-URL: https://build.opensuse.org/request/show/713773
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=94
- Update to version 18.08.05-2:
This version obsoletes:
Fix-contrib-perlapi-to-build-with-the-fix-for-CVE-2019-6438-750cc23ed.patch
- Fix spec file for older SUSE versions.
- Update to version 18.08.05:
* Add mitigation for a potential heap overflow on 32-bit systems in xmalloc.
(CVE-2019-6438, bsc#1123304).
* Other fixes:
+ Backfill - If a job has a time_limit guess the end time of a job better
if OverTimeLimit is Unlimited.
+ Fix "sacctmgr show events event=cluster"
+ Fix sacctmgr show runawayjobs from sibling cluster
+ Avoid bit offset of -1 in call to bit_nclear().
+ Insure that "hbm" is a configured GresType on knl systems.
+ Fix NodeFeaturesPlugins=node_features/knl_generic to allow other gres
other than knl.
+ cons_res: Prevent overflow on multiply.
+ Better debug for bad values in gres.conf.
+ Fix double accounting of energy at end of job.
+ Read gres.conf for cloud nodes on slurmctld.
+ Don't assume the first node of a job is the batch host when purging jobs
from a node.
+ Better debugging when a job doesn't have a job_resrcs ptr.
+ Store ave watts in energy plugins.
+ Add XCC plugin for reading Lenovo Power.
+ Fix minor memory leak when scheduling rebootable nodes.
+ Fix debug2 prefix for sched log.
+ Fix printing correct SLURM_JOB_ACCOUNT_PACK_GROUP_* in env for a Het Job.
+ sbatch - search current working directory first for job script.
OBS-URL: https://build.opensuse.org/request/show/670462
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=25
- Update to version 18.08.05:
* Add mitigation for a potential heap overflow on 32-bit systems in xmalloc.
(CVE-2019-6438, bsc#1123304).
* Other fixes:
+ Backfill - If a job has a time_limit guess the end time of a job better
if OverTimeLimit is Unlimited.
+ Fix "sacctmgr show events event=cluster"
+ Fix sacctmgr show runawayjobs from sibling cluster
+ Avoid bit offset of -1 in call to bit_nclear().
+ Insure that "hbm" is a configured GresType on knl systems.
+ Fix NodeFeaturesPlugins=node_features/knl_generic to allow other gres
other than knl.
+ cons_res: Prevent overflow on multiply.
+ Better debug for bad values in gres.conf.
+ Fix double accounting of energy at end of job.
+ Read gres.conf for cloud nodes on slurmctld.
+ Don't assume the first node of a job is the batch host when purging jobs
from a node.
+ Better debugging when a job doesn't have a job_resrcs ptr.
+ Store ave watts in energy plugins.
+ Add XCC plugin for reading Lenovo Power.
+ Fix minor memory leak when scheduling rebootable nodes.
+ Fix debug2 prefix for sched log.
+ Fix printing correct SLURM_JOB_ACCOUNT_PACK_GROUP_* in env for a Het Job.
+ sbatch - search current working directory first for job script.
+ Make it so held jobs reset the AccrueTime and do not count against any
AccrueTime limits.
+ Add SchedulerParameters option of bf_hetjob_prio=[min|avg|max] to alter
the job sorting algorithm for scheduling heterogeneous jobs.
+ Fix initialization of assoc_mgr_locks and slurmctld_locks lock
OBS-URL: https://build.opensuse.org/request/show/670322
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=83
- Update to 18.08.04, with following highlights
* Fix message sent to user to display preempted instead of time limit when
a job is preempted.
* Fix memory leak when a failure happens processing a nodes gres config.
* Improve error message when failures happen processing a nodes gres config.
* Don't skip jobs in scontrol hold.
* Allow --cpu-bind=verbose to be used with SLURM_HINT environment variable.
* Enhanced handling for runaway jobs
* cons_res: Delay exiting cr_job_test until after cores/cpus are calculated
and distributed.
* Don't check existence of srun --prolog or --epilog executables when set to
"none" and SLURM_TEST_EXEC is used.
* Add "P" suffix support to job and step tres specifications.
* Fix jobacct_gather/cgroup to work correctly when more than one task is
started on a node.
* salloc - set SLURM_NTASKS_PER_CORE and SLURM_NTASKS_PER_SOCKET in the
environment if the corresponding command line options are used.
* slurmd - fix handling of the -f flag to specify alternate config file
locations.
* Add SchedulerParameters option of bf_ignore_newly_avail_nodes to avoid
scheduling lower priority jobs on resources that become available during
the backfill scheduling cycle when bf_continue is enabled.
* job_submit/lua: Add several slurmctld return codes and add user/group info
* salloc/sbatch/srun - print warning if mutually exclusive options of --mem
and --mem-per-cpu are both set.
- Refreshed:
* pam_slurm_adopt-avoid-running-outside-of-the-sshd-PA.patch
OBS-URL: https://build.opensuse.org/request/show/663733
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=81
- restarting services on update only when activated
- added rotation of logs
- Added backported patches which harden the pam module pam_slurm_adopt
(BOO#1116758) which will be in slurm 19.05.x
* added pam_slurm_adopt-avoid-running-outside-of-the-sshd-PA.patch
[PATCH 1/3] pam_slurm_adopt: avoid running outside of the sshd PAM
* added pam_slurm_adopt-send_user_msg-don-t-copy-undefined-d.patch
[PATCH 2/3] pam_slurm_adopt: send_user_msg: don't copy undefined data
* added pam_slurm_adopt-use-uid-to-determine-whether-root-is.patch
[PATCH 3/3] pam_slurm_adopt: use uid to determine whether root is
logging on
- package slurm-pam_slurm now depends on slurm-node and not on slurm
OBS-URL: https://build.opensuse.org/request/show/657422
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=79
- Added missing perl-base dependency.
- Moved HTML docs to doc package.
- Moved config man pages to a separate package: This way, they won't
get installed on compute nodes.
- Update to 18.08.3
* Add new burst buffer state of "teardown-fail" to indicate the burst
buffer teardown operation is failing on specific buffers.
* Multiple backup slurmctld daemons can be configured
* Enable jobs with zero node count for creation and/or deletion of persistent
burst buffers.
* Add "scontrol show dwstat" command to display Cray burst buffer status.
* Add "GetSysStatus" option to burst_buffer.conf file.
* Add node and partition configuration options of "CpuBind" to control
default task binding.
* Add "NumaCpuBind" option to knl.conf
* Add sbatch "--batch" option to identify features required on batch node.
* Add "BatchFeatures" field to output of "scontrol show job".
* Add support for "--bb" option to sbatch command.
* Add new SystemComment field to job data structure and database.
* Expand reservation "flags" field from 32 to 64 bits.
* Add job state flag of "SIGNALING" to avoid race condition.
* Properly handle srun --will-run option when there are jobs in COMPLETING
state.
* Properly report who is signaling a step.
* Don't combine updated reservation records in sreport's reservation report.
* node_features plugin - Add suport for XOR & XAND of job constraints (node
feature specifications).
OBS-URL: https://build.opensuse.org/request/show/650545
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=75
- Update to 17.11.9
* Fix segfault in slurmctld when a job's node bitmap is NULL during a
scheduling cycle. Primarily caused by EnforcePartLimits=ALL.
* Remove erroneous unlock in acct_gather_energy/ipmi.
* Enable support for hwloc version 2.0.1.
* Fix 'srun -q' (--qos) option handling.
* Fix socket communication issue that can lead to lost task completition
messages, which will cause a permanently stuck srun process.
* Handle creation of TMPDIR if environment variable is set or changed in
a task prolog script.
* Avoid node layout fragmentation if running with a fixed CPU count but
without Sockets and CoresPerSocket defined.
* burst_buffer/cray - Fix datawarp swap default pool overriding jobdw.
* Fix incorrect job priority assignment for multi-partition job with
different PriorityTier settings on the partitions.
* Fix sinfo to print correct node state.
- When using a remote shared StateSaveLocation, slurmctld needs to
be started after remote filesystems have become available.
Add 'remote-fs.target' to the 'After=' directive in slurmctld.service
(boo#1103561).
- Update to 17.11.8
* Fix incomplete RESPONSE_[RESOURCE|JOB_PACK]_ALLOCATION building path.
* Do not allocate nodes that were marked down due to the node not responding
by ResumeTimeout.
* task/cray plugin - search for "mems" cgroup information in the file
"cpuset.mems" then fall back to the file "mems".
* Fix ipmi profile debug uninitialized variable.
* PMIx: fixed the direct connect inline msg sending.
OBS-URL: https://build.opensuse.org/request/show/629222
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=64
- Update from 17.11.5 to 17.11.7
- Fix security issue in handling of username and gid fields
CVE-2018-10995 and bsc#1095508 what implied an
update from 17.11.5 to 17.11.7
Highlights of 17.11.6:
* CRAY - Add slurmsmwd to the contribs/cray dir
* PMIX - Added the direct connect authentication.
* Prevent the backup slurmctld from losing the active/available node
features list on takeover.
* Be able to force power_down of cloud node even if in power_save state.
* Allow cloud nodes to be recognized in Slurm when booted out of band.
* Numerous fixes - check 'NEWS' file.
Highlights of 17.11.7:
* Notify srun and ctld when unkillable stepd exits.
* Numerous fixes - check 'NEWS' file.
- Add: slurmsmwd-uses-xdaemon_-for-systemd.patch
* Fixes daemoniziation in newly introduced slurmsmwd daemon.
- Rename:
split-xdaemon-in-xdaemon_init-and-xdaemon_finish-for-systemd-compatibilty.patch
to split-xdaemon-in-xdaemon_init-and-xdaemon_finish-for.patch
* remain in sync with commit messages which introduced that file
OBS-URL: https://build.opensuse.org/request/show/616050
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=14
- Fix security issue in handling of username and gid fields
CVE-2018-10995 and bsc#1095508 what implied an
update from 17.11.5 to 17.11.7
- renanmed split-xdaemon-in-xdaemon_init-and-xdaemon_finish-for-systemd-compatibilty.patch
to split-xdaemon-in-xdaemon_init-and-xdaemon_finish-for.patch
in order to be in sync with commit messages which introduced that file
OBS-URL: https://build.opensuse.org/request/show/616031
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=59
- Fix security issue in handling of username and gid fields
CVE-2018-10995 what implied an update from 17.11.5 to 17.11.7
- Update from 17.11.5 to 17.11.7
Highlights of 17.11.6:
* CRAY - Add slurmsmwd to the contribs/cray dir
* PMIX - Added the direct connect authentication.
* Prevent the backup slurmctld from losing the active/available node
features list on takeover.
* Be able to force power_down of cloud node even if in power_save state.
* Allow cloud nodes to be recognized in Slurm when booted out of band.
* Numerous fixes - check 'NEWS' file.
Highlights of 17.11.7:
* Notify srun and ctld when unkillable stepd exits.
* Numerous fixes - check 'NEWS' file.
OBS-URL: https://build.opensuse.org/request/show/615950
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=58
- Add requires for slurm-sql to the slurmdbd package.
- Package READMEs for pam and pam_slurm_adopt.
- Use the new %%license directive for COPYING file.
- Add:
* split-xdaemon-in-xdaemon_init-and-xdaemon_finish-for-systemd-compatibilty.patch
* slurmctld-uses-xdaemon_-for-systemd.patch
* slurmd-uses-xdaemon_-for-systemd.patch
* slurmdbd-uses-xdaemon_-for-systemd.patch
* removed-deprecated-xdaemon.patch
Fix interaction with systemd: systemd expects that a
daemonizing process doesn't go away until the PID file
with it PID of the daemon has bee written (bsc#1084125).
- Make sure systemd services get restarted only when all
packages are in a consistent state, not in the middle
of an 'update' transaction (bsc#1088693).
Since the %postun scripts that run on update are from
the old package they cannot be changed - thus we work
around the restart breakage.
(bsc#1086859).
OBS-URL: https://build.opensuse.org/request/show/596387
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=12
packages are in a consistent state, not in the middle
of an 'update' transaction (bsc#1088693).
Since the %postun scripts that run on update are from
the old package they cannot be changed - thus we work
around the restart breakage.
(bsc#1086859).
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=51
- Fix security issue in accounting_storage/mysql plugin by always escaping
strings within the slurmdbd. CVE-2018-7033
http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2018-7033
(bsc#1085240).
- Update slurm to v17.11.5 (FATE#325451)
Highlights of 17.11:
* Support for federated clusters to manage a single work-flow
across a set of clusters.
* Support for heterogeneous job allocations (various processor types,
memory sizes, etc. by job component). Support for heterogeneous job
steps within a single MPI_COMM_WORLD is not yet supported for most
configurations.
* X11 support is now fully integrated with the main Slurm code. Remove
any X11 plugin configured in your plugstack.conf file to avoid errors
being logged about conflicting options.
* Added new advanced reservation flag of "flex", which permits jobs
requesting the reservation to begin prior to the reservation's
start time and use resources inside or outside of the reservation.
A typical use case is to prevent jobs not explicitly requesting the
reservation from using those reserved resources rather than forcing
jobs requesting the reservation to use those resources in the time
frame reserved.
* The sprio command has been modified to report a job's priority
information for every partition the job has been submitted to.
* Group ID lookup performed at job submit time to avoid lookup on
all compute nodes. Enable with PrologFlags=SendGIDs configuration
parameter.
* Slurm commands and daemons dynamically link to libslurmfull.so
instead of statically linking. This dramatically reduces the
footprint of Slurm.
OBS-URL: https://build.opensuse.org/request/show/587617
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=46
- moved config files to slurm-config package (FATE#324574).
- Moved slurmstepd and man page into slurm-node due to slurmd dependency
- Moved config files into slurm-node
- Moved slurmd rc scripts into slurm-node
- Made slurm-munge require slurm-plugins instead of slurm itself
- slurm-node suggested slurm-munge, causing the whole slurm to be
installed. The slurm-plugins seems to be a more base class
(FATE#324574).
- split up light wight slurm-node package for deployment on nodes
(FATE#324574).
OBS-URL: https://build.opensuse.org/request/show/571153
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=9
- moved config files to slurm-config package
- Moved slurmstepd and man page into slurm-node due to slurmd dependency
- Moved config files into slurm-node
- Moved slurmd rc scripts into slurm-node
- Made slurm-munge require slurm-plugins instead of slurm itself
- slurm-node suggested slurm-munge, causing the whole slurm to be
installed. The slurm-plugins seems to be a more base class
- split up light wight slurm-node package for deployment on nodes
OBS-URL: https://build.opensuse.org/request/show/570889
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=38
- Updated to 17.02.9 to fix CVE-2017-15566 (bsc#1065697).
Changes in 17.0.9
* When resuming powered down nodes, mark DOWN nodes right after
ResumeTimeout
has been reached (previous logic would wait about one minute longer).
* Fix sreport not showing full column name for TRES Count.
* Fix slurmdb_reservations_get() giving wrong usage data when job's spanned
reservation that was modified.
* Fix sreport reservation utilization report showing bad data.
* Show all TRES' on a reservation in sreport reservation utilization report
by default.
* Fix sacctmgr show reservation handling "end" parameter.
* Work around issue with sysmacros.h and gcc7 / glibc 2.25.
* Fix layouts code to only allow setting a boolean.
* Fix sbatch --wait to keep waiting even if a message timeout occurs.
* CRAY - If configured with NodeFeatures=knl_cray and there are non-KNL
nodes which include no features the slurmctld will abort without
this patch when attemping strtok_r(NULL).
* Fix regression in 17.02.7 which would run the spank_task_privileged as
part of the slurmstepd instead of it's child process.
* Fix security issue in Prolog and Epilog by always prepending SPANK_ to
all user-set environment variables. CVE-2017-15566.
Changes in 17.0.8:
* Add 'slurmdbd:' to the accounting plugin to notify message is from dbd
instead of local.
* mpi/mvapich - Buffer being only partially cleared. No failures observed.
* Fix for job --switch option on dragonfly network.
* In salloc with --uid option, drop supplementary groups before changing UID.
* jobcomp/elasticsearch - strip any trailing slashes from JobCompLoc.
* jobcomp/elasticsearch - fix memory leak when transferring generated buffer. (forwarded request 538161 from eeich)
OBS-URL: https://build.opensuse.org/request/show/538162
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/slurm?expand=0&rev=5
- Updated to 17.02.9 to fix CVE-2017-15566 (bsc#1065697).
Changes in 17.0.9
* When resuming powered down nodes, mark DOWN nodes right after
ResumeTimeout
has been reached (previous logic would wait about one minute longer).
* Fix sreport not showing full column name for TRES Count.
* Fix slurmdb_reservations_get() giving wrong usage data when job's spanned
reservation that was modified.
* Fix sreport reservation utilization report showing bad data.
* Show all TRES' on a reservation in sreport reservation utilization report
by default.
* Fix sacctmgr show reservation handling "end" parameter.
* Work around issue with sysmacros.h and gcc7 / glibc 2.25.
* Fix layouts code to only allow setting a boolean.
* Fix sbatch --wait to keep waiting even if a message timeout occurs.
* CRAY - If configured with NodeFeatures=knl_cray and there are non-KNL
nodes which include no features the slurmctld will abort without
this patch when attemping strtok_r(NULL).
* Fix regression in 17.02.7 which would run the spank_task_privileged as
part of the slurmstepd instead of it's child process.
* Fix security issue in Prolog and Epilog by always prepending SPANK_ to
all user-set environment variables. CVE-2017-15566.
Changes in 17.0.8:
* Add 'slurmdbd:' to the accounting plugin to notify message is from dbd
instead of local.
* mpi/mvapich - Buffer being only partially cleared. No failures observed.
* Fix for job --switch option on dragonfly network.
* In salloc with --uid option, drop supplementary groups before changing UID.
* jobcomp/elasticsearch - strip any trailing slashes from JobCompLoc.
* jobcomp/elasticsearch - fix memory leak when transferring generated buffer.
OBS-URL: https://build.opensuse.org/request/show/538161
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=29
- Updated to slurm 17-02-7-1
* Added python as BuildRequires
* Removed sched-wiki package
* Removed slurmdb-direct package
* Obsoleted sched-wiki and slurmdb-direct packages
* Removing Cray-specific files
* Added /etc/slurm/layout.d files (new for this version)
* Remove /etc/slurm/cgroup files from package
* Added lib/slurm/mcs_account.so
* Removed lib/slurm/jobacct_gather_aix.so
* Removed lib/slurm/job_submit_cnode.so
- Created slurm-sql package
- Moved files from slurm-plugins to slurm-torque package
- Moved creation of /usr/lib/tmpfiles.d/slurm.conf into slurm.spec
* Removed tmpfiles.d-slurm.conf
- Made tmpfiles_create post-install macro SLE12 SP2 or greater
- Directly calling systemd-tmpfiles --create for before SLE12 SP2
- Allows OpenSUSE Factory build as well
- Removes unused .service files from project
- Adds /var/run/slurm to /usr/lib/tmpfiles.d for boottime creation
- Patches upstream .service files to allow for /var/run/slurm path
- Modifies slurm.conf to allow for /var/run/slurm path
OBS-URL: https://build.opensuse.org/request/show/531499
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=21
- Updated to 16.05.8.1
* Remove StoragePass from being printed out in the slurmdbd log at debug2
level.
* Defer PATH search for task program until launch in slurmstepd.
* Modify regression test1.89 to avoid leaving vestigial job. Also reduce
logging to reduce likelyhood of Expect buffer overflow.
* Do not PATH search for mult-prog launches if LaunchParamters=test_exec is
enabled.
* Fix for possible infinite loop in select/cons_res plugin when trying to
satisfy a job's ntasks_per_core or socket specification.
* If job is held for bad constraints make it so once updated the job doesn't
go into JobAdminHeld.
* sched/backfill - Fix logic to reserve resources for jobs that require a
node reboot (i.e. to change KNL mode) in order to start.
* When unpacking a node or front_end record from state and the protocol
version is lower than the min version, set it to the min.
* Remove redundant lookup for part_ptr when updating a reservation's nodes.
* Fix memory and file descriptor leaks in slurmd daemon's sbcast logic.
* Do not allocate specialized cores to jobs using the --exclusive option.
* Cancel interactive job if Prolog failure with "PrologFlags=contain" or
"PrologFlags=alloc" configured. Send new error prolog failure message to
the salloc or srun command as needed.
* Prevent possible out-of-bounds read in slurmstepd on an invalid #! line.
* Fix check for PluginDir within slurmctld to work with multiple directories.
* Cancel interactive jobs automatically on communication error to launching
srun/salloc process.
* Fix security issue caused by insecure file path handling triggered by the
failure of a Prolog script. To exploit this a user needs to anticipate or
cause the Prolog to fail for their job. CVE-2016-10030 (bsc#1018371).
- Replace group/user add macros with function calls.
- Disable building with netloc support: the netloc API is part of the devel
branch of hwloc. Since this devel branch was included accidentally and has
been reversed since, we need to disable this for the time being.
- Conditionalized architecture specific pieces to support non-x86 architectures
better.
- Remove: unneeded 'BuildRequires: python'
- Add:
BuildRequires: freeipmi-devel
BuildRequires: libibmad-devel
BuildRequires: libibumad-devel
so they are picked up by the slurm build.
- Enable modifications from openHPC Project.
- Enable lua API package build.
- Add a recommends for slurm-munge to the slurm package:
This is way, the munge auth method is available and slurm
works out of the box.
- Create /var/lib/slurm as StateSaveLocation directory.
/tmp is dangerous.
- Keep %{_libdir}/libpmi* and %{_libdir}/mpi_pmi2* on SUSE.
OBS-URL: https://build.opensuse.org/request/show/454272
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=13
- Fix build with and without OHCP_BUILD define.
- Fix build for systemd and non-systemd.
- Updated to 16-05-5 - equvalent to OpenHPC 1.2.
* Fix issue with resizing jobs and limits not be kept track of correctly.
* BGQ - Remove redeclaration of job_read_lock.
* BGQ - Tighter locks around structures when nodes/cables change state.
* Make it possible to change CPUsPerTask with scontrol.
* Make it so scontrol update part qos= will take away a partition QOS from
a partition.
* Backfill scheduling properly synchronized with Cray Node Health Check.
Prior logic could result in highest priority job getting improperly
postponed.
* Make it so daemons also support TopologyParam=NoInAddrAny.
* If scancel is operating on large number of jobs and RPC responses from
slurmctld daemon are slow then introduce a delay in sending the cancel job
requests from scancel in order to reduce load on slurmctld.
* Remove redundant logic when updating a job's task count.
* MySQL - Fix querying jobs with reservations when the id's have rolled.
* Perl - Fix use of uninitialized variable in slurm_job_step_get_pids.
* Launch batch job requsting --reboot after the boot completes.
* Do not attempt to power down a node which has never responded if the
slurmctld daemon restarts without state.
* Fix for possible slurmstepd segfault on invalid user ID.
* MySQL - Fix for possible race condition when archiving multiple clusters
at the same time.
* Add logic so that slurmstepd can be launched under valgrind.
* Increase buffer size to read /proc/*/stat files.
* Remove the SchedulerParameters option of "assoc_limit_continue", making it
the default value. Add option of "assoc_limit_stop". If "assoc_limit_stop"
is set and a job cannot start due to association limits, then do not attempt
to initiate any lower priority jobs in that partition. Setting this can
decrease system throughput and utlization, but avoid potentially starving
larger jobs by preventing them from launching indefinitely.
* Update a node's socket and cores per socket counts as needed after a node
boot to reflect configuration changes which can occur on KNL processors.
Note that the node's total core count must not change, only the distribution
of cores across varying socket counts (KNL NUMA nodes treated as sockets by
Slurm).
* Rename partition configuration from "Shared" to "OverSubscribe". Rename
salloc, sbatch, srun option from "--shared" to "--oversubscribe". The old
options will continue to function. Output field names also changed in
scontrol, sinfo, squeue and sview.
* Add SLURM_UMASK environment variable to user job.
* knl_conf: Added new configuration parameter of CapmcPollFreq.
* Cleanup two minor Coverity warnings.
* Make it so the tres units in a job's formatted string are converted like
they are in a step.
* Correct partition's MaxCPUsPerNode enforcement when nodes are shared by
multiple partitions.
* node_feature/knl_cray - Prevent slurmctld GRES errors for "hbm" references.
* Display thread name instead of thread id and remove process name in stderr
logging for "thread_id" LogTimeFormat.
* Log IP address of bad incomming message to slurmctld.
* If a user requests tasks, nodes and ntasks-per-node and
tasks-per-node/nodes != tasks print warning and ignore ntasks-per-node.
* Release CPU "owner" file locks.
* Update seff to fix warnings with ncpus, and list slurm-perlapi dependency
in spec file.
* Allow QOS timelimit to override partition timelimit when EnforcePartLimits
is set to all/any.
* Make it so qsub will do a "basename" on a wrapped command for the output
and error files.
* Add logic so that slurmstepd can be launched under valgrind.
* Increase buffer size to read /proc/*/stat files.
* Prevent job stuck in configuring state if slurmctld daemon restarted while
PrologSlurmctld is running. Also re-issue burst_buffer/pre-load operation
as needed.
* Move test for job wait reason value of BurstBufferResources and
BurstBufferStageIn later in the scheduling logic.
* Document which srun options apply to only job, only step, or job and step
allocations.
* Use more compatible function to get thread name (>= 2.6.11).
* Make it so the extern step uses a reverse tree when cleaning up.
* If extern step doesn't get added into the proctrack plugin make sure the
sleep is killed.
* Add web links to Slurm Diamond Collectors (from Harvard University) and
collectd (from EDF).
* Add job_submit plugin for the "reboot" field.
* Make some more Slurm constants (INFINITE, NO_VAL64, etc.) available to
job_submit/lua plugins.
* Send in a -1 for a taskid into spank_task_post_fork for the extern_step.
* MYSQL - Sightly better logic if a job completion comes in with an end time
of 0.
* task/cgroup plugin is configured with ConstrainRAMSpace=yes, then set soft
memory limit to allocated memory limit (previously no soft limit was set).
* Streamline when schedule() is called when running with message aggregation
on batch script completes.
* Fix incorrect casting when [un]packing derived_ec on slurmdb_job_rec_t.
* Document that persistent burst buffers can not be created or destroyed using
the salloc or srun --bb options.
* Add support for setting the SLURM_JOB_ACCOUNT, SLURM_JOB_QOS and
SLURM_JOB_RESERVAION environment variables are set for the salloc command.
Document the same environment variables for the salloc, sbatch and srun
commands in their man pages.
* Fix issue where sacctmgr load cluster.cfg wouldn't load associations
that had a partition in them.
* Don't return the extern step from sstat by default.
* In sstat print 'extern' instead of 4294967295 for the extern step.
* Make advanced reservations work properly with core specialization.
* slurmstepd modified to pre-load all relevant plugins at startup to avoid
the possibility of modified plugins later resulting in inconsistent API
or data structures and a failure of slurmstepd.
* Export functions from parse_time.c in libslurm.so.
* Export unit convert functions from slurm_protocol_api.c in libslurm.so.
* Fix scancel to allow multiple steps from a job to be cancelled at once.
* Update and expand upgrade guide (in Quick Start Administrator web page).
* burst_buffer/cray: Requeue, but do not hold a job which fails the pre_run
operation.
* Insure reported expected job start time is not in the past for pending jobs.
* Add support for PMIx v2.
OBS-URL: https://build.opensuse.org/request/show/441490
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=12
- version 15.08.7.1
* Remove the 1024-character limit on lines in batch scripts.
task/affinity: Disable core-level task binding if more CPUs required than
available cores.
* Preemption/gang scheduling: If a job is suspended at slurmctld restart or
reconfiguration time, then leave it suspended rather than resume+suspend.
* Don't use lower weight nodes for job allocation when topology/tree used.
* Don't allow user specified reservation names to disrupt the normal
reservation sequeuece numbering scheme.
* Avoid hard-link/copy of script/environment files for job arrays. Use the
master job record file for all tasks of the job array.
NOTE: Job arrays submitted to Slurm version 15.08.6 or later will fail if
the slurmctld daemon is downgraded to an earlier version of Slurm.
* In slurmctld log file, log duplicate job ID found by slurmd. Previously was
being logged as prolog/epilog failure.
* If a job is requeued while in the process of being launch, remove it's
job ID from slurmd's record of active jobs in order to avoid generating a
duplicate job ID error when launched for the second time (which would
drain the node).
* Cleanup messages when handling job script and environment variables in
older directory structure formats.
* Prevent triggering gang scheduling within a partition if configured with
PreemptType=partition_prio and PreemptMode=suspend,gang.
* Decrease parallelism in job cancel request to prevent denial of service
when cancelling huge numbers of jobs.
* If all ephemeral ports are in use, try using other port numbers.
* Prevent "scontrol update job" from updating jobs that have already finished.
* Show requested TRES in "squeue -O tres" when job is pending.
* Backfill scheduler: Test association and QOS node limits before reserving
resources for pending job.
* Many bug fixes.
- Use source services to download package.
- Fix code for new API of hwloc-2.0.
- package netloc_to_topology where avialable.
- Package documentation.
OBS-URL: https://build.opensuse.org/request/show/435622
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=10
- version 14.03.6
* Added support for native Slurm operation on Cray systems (without ALPS).
* Added partition configuration parameters AllowAccounts, AllowQOS, DenyAccounts and DenyQOS to provide greater control over use.
* Added the ability to perform load based scheduling. Allocating resources to jobs on the nodes with the largest number if idle CPUs.
* Added support for reserving cores on a compute node for system services (core specialization)
* Add mechanism for job_submit plugin to generate error message for srun, salloc or sbatch to stderr.
* Support for Postgres database has long since been out of date and problematic, so it has been removed entirely. If you would like to use it the code still exists in <= 2.6, but will not be included in this and future versions of the code.
* Added new structures and support for both server and cluster resources.
* Significant performance improvements, especially with respect to job array support.
- update files list
OBS-URL: https://build.opensuse.org/request/show/242503
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=5
- update to version 2.6.7
* Support for job arrays, which increases performance and ease of
use for sets of similar jobs.
* Job profiling capability added to record a wide variety of job
characteristics for each task on a user configurable periodic
basis. Data currently available includes CPU use, memory use,
energy use, Infiniband network use, Lustre file system use, etc.
* Support for MPICH2 using PMI2 communications interface with much
greater scalability.
* Prolog and epilog support for advanced reservations.
* Much faster throughput for job step execution with --exclusive
option. The srun process is notified when resources become
available rather than periodic polling.
* Support improved for Intel MIC (Many Integrated Core) processor.
* Advanced reservations with hostname and core counts now supports
asymmetric reservations (e.g. specific different core count for
each node).
* External sensor plugin infrastructure added to record power
consumption, temperature, etc.
* Improved performance for high-throughput computing.
* MapReduce+ support (launches ~1000x faster, runs ~10x faster).
* Added "MaxCPUsPerNode" partition configuration parameter. This
can be especially useful to schedule GPUs. For example a node
can be associated with two Slurm partitions (e.g. "cpu" and
"gpu") and the partition/queue "cpu" could be limited to only a
subset of the node's CPUs, insuring that one or more CPUs would
be available to jobs in the "gpu" partition/queue.
OBS-URL: https://build.opensuse.org/request/show/226317
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=4
- version 2.5.7
* Fix for linking to the select/cray plugin to not give warning
about undefined variable.
* Add missing symbols to the xlator.h
* Avoid placing pending jobs in AdminHold state due to backfill
scheduler interactions with advanced reservation.
* Accounting - make average by task not cpu.
* POE - Correct logic to support poe option "-euidevice sn_all"
and "-euidevice sn_single".
* Accounting - Fix minor initialization error.
* POE - Correct logic to support srun network instances count
with POE.
* POE - With the srun --launch-cmd option, report proper task
count when the --cpus-per-task option is used without the
--ntasks option.
* POE - Fix logic binding tasks to CPUs.
* sview - Fix race condition where new information could of
slipped past the node tab and we didn't notice.
* Accounting - Fix an invalid memory read when slurmctld sends
data about start job to slurmdbd.
* If a prolog or epilog failure occurs, drain the node rather
than setting it down and killing all of its jobs.
* Priority/multifactor - Avoid underflow in half-life calculation.
* POE - pack missing variable to allow fanout (more than 32
nodes)
* Prevent clearing reason field for pending jobs. This bug was
introduced in v2.5.5 (see "Reject job at submit time ...").
* BGQ - Fix issue with preemption on sub-block jobs where a job
would kill all preemptable jobs on the midplane instead of just
the ones it needed to.
OBS-URL: https://build.opensuse.org/request/show/177944
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=3
- version 2.5.4
* Support for Intel® Many Integrated Core (MIC) processors.
* User control over CPU frequency of each job step.
* Recording power usage information for each job.
* Advanced reservation of cores rather than whole nodes.
* Integration with IBM's Parallel Environment including POE (Parallel Operating Environment) and NRT (Network Resource Table) API.
* Highly optimized throughput for serial jobs in a new "select/serial" plugin.
* CPU load is information available
* Configurable number of CPUs available to jobs in each SLURM partition, which provides a mechanism to reserve CPUs for use with GPUs.
OBS-URL: https://build.opensuse.org/request/show/163479
OBS-URL: https://build.opensuse.org/package/show/network:cluster/slurm?expand=0&rev=2
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
