- Update to v1.6.2
* Set base Ceph operator image and example deployments to v16.2.2
* Update snapshot APIs from v1beta1 to v1
* Documentation for creating static PVs
* Allow setting primary-affinity for the OSD
* Remove unneeded debug log statements
* Preserve volume claim template annotations during upgrade
* Allow re-creating erasure coded pool with different settings
* Double mon failover timeout during a node drain
* Remove unused volumesource schema from CephCluster CRD
* Set the device class on raw mode osds
* External cluster schema fix to allow not setting mons
* Add phase to the CephFilesystem CRD
* Generate full schema for volumeClaimTemplates in the CephCluster CRD
* Automate upgrades for the MDS daemon to properly scale down and scale up
* Add Vault KMS support for object stores
* Ensure object store endpoint is initialized when creating an object user
* Support for OBC operations when RGW is configured with TLS
* Preserve the OSD topology affinity during upgrade for clusters on PVCs
* Unify timeouts for various Ceph commands
* Allow setting annotations on RGW service
* Expand PVC size of mon daemons if requested
- Update to v1.6.1
* Disable host networking by default in the CSI plugin with option to enable
* Fix the schema for erasure-coded pools so replication size is not required
* Improve node watcher for adding new OSDs
* Operator base image updated to v16.2.1
* Deployment examples updated to Ceph v15.2.11
* Update Ceph-CSI to v3.3.1
* Allow any device class for the OSDs in a pool instead of restricting the schema
* Fix metadata OSDs for Ceph Pacific
* Allow setting the initial CRUSH weight for an OSD
* Fix object store health check in case SSL is enabled
* Upgrades now ensure latest config flags are set for MDS and RGW
* Suppress noisy RGW log entry for radosgw-admin commands
- Update to v1.6.0
* Removed Storage Providers
* CockroachDB
* EdgeFS
* YugabyteDB
* Ceph
* Support for creating OSDs via Drive Groups was removed.
* Ceph Pacific (v16) support
* CephFilesystemMirror CRD to support mirroring of CephFS volumes with Pacific
* Ceph CSI Driver
* CSI v3.3.0 driver enabled by default
* Volume Replication Controller for improved RBD replication support
* Multus support
* GRPC metrics disabled by default
* Ceph RGW
* Extended the support of vault KMS configuration
* Scale with multiple daemons with a single deployment instead of a separate deployment for each rgw daemon
* OSDs
* LVM is no longer used to provision OSDs
* More efficient updates for multiple OSDs at the same time
* Multiple Ceph mgr daemons are supported for stretch clusters
and other clusters where HA of the mgr is critical (set count: 2 under mgr in the CephCluster CR)
* Pod Disruption Budgets (PDBs) are enabled by default for Mon,
RGW, MDS, and OSD daemons. See the disruption management settings.
* Monitor failover can be disabled, for scenarios where
maintenance is planned and automatic mon failover is not desired
* CephClient CRD has been converted to use the controller-runtime library
OBS-URL: https://build.opensuse.org/request/show/892260
OBS-URL: https://build.opensuse.org/package/show/filesystems:ceph/rook?expand=0&rev=95
- Update to v1.5.8
* Ceph
* Update Ceph-CSI to v3.2.1 (#7506)
* Use latest Ceph API for setting dashboard and rgw credentials (#7641)
* Redact secret info from reconcile diffs in debug logs (#7630)
* Continue to get available devices if failed to get a device info (#7608)
* Include RGW pods in list for rescheduling from failed node (#7537)
* Enforce pg_auto_scaler on rgw pools (#7513)
* Prevent voluntary mon drain while another mon is failing over (#7442)
* Avoid restarting all encrypted OSDs on cluster growth (#7489)
* Set secret type on external cluster script (#7473)
* Fix init container "expand-encrypted-bluefs" for encrypted OSDs (#7466)
* Fail pool creation if the sub failure domain is the same as the failure domain (#7284)
* Set default backend for vault and remove temp key for encrypted OSDs (#7454)
OBS-URL: https://build.opensuse.org/request/show/887308
OBS-URL: https://build.opensuse.org/package/show/filesystems:ceph/rook?expand=0&rev=91
- Update to v1.5.7
* Ceph
* CSI Troubleshooting Guide (#7157)
* Print device information in OSD prepare logs (#7194)
* Expose vault curl error in the OSD init container for KCS configurations (#7193)
* Prevent re-using a device to configure an OSD on PVC from a previous cluster (#7170)
* Remove crash collector if all Ceph pods moved off a node (#7160)
* Add helm annotation to keep CRDs in the helm chart during uninstall (#7162)
* Bind mgr modules to all interfaces instead of pod ip (#7151)
* Check for orchestration cancellation while waiting for all OSDs to start (#7112)
* Skip pdb reconcile on create and delete events (#7155)
* Silence harmless errors in log when the operator is still initializing (#7056)
* Add --extra-create-metadata flag to the CSI driver (#7147)
* Add deviceClass to the object store schema (#7132)
* Simplify the log-collector container name (#7133)
* Skip csi detection if CSI is disabled (#6866)
* Remove Rook pods stuck in terminating state on a failed node (#6999)
* Timeout for rgw configuration to prevent stuck object store when no healthy OSDs (#7075)
* Update lib bucket provisioner for OBCs (#7086)
- Drop csi-images-SUSE.patch
OBS-URL: https://build.opensuse.org/request/show/876712
OBS-URL: https://build.opensuse.org/package/show/filesystems:ceph/rook?expand=0&rev=89
- Derive CSI and sidecar image versions from code defaults rather
than images found in the build service
- Update to v1.4.7
* Ceph
* Log warning about v14.2.13 being an unsupported Ceph version due to
errors creating new OSDs (#6545)
* Disaster recovery guide for PVCs (#6452)
* Set the deviceClass for OSDs in non-PVC clusters (#6545)
* External cluster script to fail if prometheus port is not default (#6504)
* Remove the osd pvc from the osd purge job (#6533)
* External cluster script added additional checks for monitoring
endpoint (#6473)
* Ignore Ceph health error MDS_ALL_DOWN during reconciliation (#6494)
* Add optional labels to mon pods (#6515)
* Assert type for logging errors before using it (#6503)
* Check for orphaned mon resources with every reconcile (#6493)
* Update the mon PDBs if the maxUnavailable changed (#6469)
* NFS
* Update documentation and examples (#6455)
- Drop OFFSET from cephcsi image tag
- Update helm chart to use appropriate version prefix for the final registry
destination (e.g. registry.suse.com or registry.opensuse.org)
- Improve consistency with image tags
- Update to v1.4.6
* Support IPv6 single-stack (#6283)
* Only start a single CSI provisioner in single-node test clusters (#6437)
OBS-URL: https://build.opensuse.org/request/show/850543
OBS-URL: https://build.opensuse.org/package/show/filesystems:ceph/rook?expand=0&rev=88
- Update to v1.4.5
* Update the CSI driver to v3.1.1 (#6340)
* Fix drive group deployment failure (#6267)
* Fix OBC upgrade from 1.3 to 1.4 external cluster (#6353)
* Remove user unlink while deleting OBC (#6338)
* Enable RBAC in the helm chart for enabling monitoring (#6352)
* Disable encryption keyring parameter not necessary after
opening block (#6350)
* Improve reconcile performance in clusters with many OSDs on
PVCs (#6330)
* Only one external cluster secret supported in import script (#6343)
* Allow OSD PVC template name to be set to any value (#6307)
* OSD prepare job was failing due to low aio-max-nr setting (#6284)
* During upgrade assume a pod spec changed if diff checking fails (#6272)
* Merge config from rook-config-override configmap to the default global
config file (#6252)
- Package all sample yaml files in rook-k8s-yaml
- Update helm chart version to match rook product version plus
the current release number
- Update to v1.4.4
* Upgrade to v1.4.3 for cluster-on-pvc hung due to changing label
selectors on the mons (#6256)
* Remove osd status configmap for nodes with long names (#6235)
* Allow running rgw daemons from an external cluster (#6226)
- Create symlinks in /usr/local/bin for toolbox.sh and rook to
ensure compatibility with upstream sample yamls
- Removed orchestrator-cli-rename.patch as it got merged
OBS-URL: https://build.opensuse.org/request/show/839222
OBS-URL: https://build.opensuse.org/package/show/filesystems:ceph/rook?expand=0&rev=86
- fixed spec-file:
* operator.yaml does not get changed to use the SUSE-images
- helm chart, manifests:
* fixed tolerations
* Update SUSE documentation URL in NOTES.txt
- ceph: fix drive group deployment failure (bsc#1176170)
- helm chart, manifests:
* Add tolerations to cluster & CRDs
* Require kubeVersion >= 1.11
* Use rbac.authorization.k8s.io/v1
* Add affinities for label schema
* Set Rook log level to DEBUG
* Remove FlexVolume agent
* Require currentNamespaceOnly=true
* Replace NOTES.txt with SUSE specific version
- Include operator and common yamls in manifest package
- Update to v1.4.3
* The Ceph-CSI driver was being unexpectedly removed by the garbage
collector in some clusters. For more details to apply a fix during
the upgrade to this patch release, see these steps. (#616)
* Add storageClassDeviceSet label to osd pods (#6225)
* DNS suffix issue for OBCs in custom DNS suffix clusters (#6234)
* Cleanup mon canary pvc if the failover failed (#6224)
* Only enable mgr init container if the dashboard is enabled (#6198)
* cephobjectstore monitoring goroutine must be stopped during
uninstall (#6208)
* Remove NParts and Cache_Size from MDCACHE block in the NFS
configuration (#6207)
* Purge a down osd with a job created by the admin (#6127)
* Do not use label selector on external mgr service (#6142)
* Allow uninstall even if volumes still exist with a new CephCluster
setting (#6145)
Fri Sep 10 21:16:34 UTC 2020 - Mike Latimer <mlatimer@suse.com>
- Update to v1.4.2
- Patch release focusing on small feature additions and bug fixes.
* Improve check for LVM on the host to allow installing of OSDs (#6175)
* Set the OSD prepare resource limits (#6118)
* Allow memory limits below recommended settings (#6116)
* Use full DNS suffix for object endpoint with OBCs (#6170)
* Remove the CSI driver lifecycle preStop hook (#6141)
* External cluster optional settings for provisioners (#6048)
* Operator watches nodes that match OSD placement rules (#6156)
* Allow user to add labels to the cluster daemon pods (#6084#6082)
* Fix vulnerability in package golang.org/x/text (#6136)
* Add expansion support for encrypted osd on pvc (#6126)
* Do not use realPath for OSDs on PVCs (#6120, @leseb)
* Example object store manifests updated for consistency (#6123)
* Separate topology spread constrinats for osd prepare jobs and
osd daemons (#6103)
* Pass CSI resources as strings in the helm chart (#6104)
* Improve callCephVolume() for list and prepare (#6059)
* Improved multus support for the CSI driver configuration (#5740)
* Object store healthcheck yaml examples (#6090)
* Add support for wal encrypted device on pvc (#6062)
* Updated helm usage in documentation (#6086)
* More details for RBD Mirroring documentation (#6083)
- Build process changes:
- Set CSI sidecar versions through _service, and set all versions in
code through a single patch file
+ csi-images-SUSE.patch
- csi-dummy-images.patch
- Use github.com/SUSE/rook and suse-release-1.4 tag in update.sh
- Create module dependencies through _service, and store these dependencies
in vendor.tar.gz (replacing rook-[version]-vendor.tar.xz)
- Modify build commands to include "-mod=vendor" to use new vendor tarball
- Add CSI sidecars as BuildRequires, in order to determine versions through
_service process
- Replace %setup of vendor tarball with a simple tar extraction
- Move registry detection to %prep, and set correct registry through a
search and replace on the SUSE_REGISTRY string
- Use variables to track rook, ceph and cephcsi versions
- Add '#!BuildTag', and 'appVersion' to chart.yaml
- Add required versioning to helm chart
- Leave ceph-csi templates in /etc, and include them in main rook package.
- csi-template-paths.patch
- Include only designated yaml examples in rook-k8s-yaml package
OBS-URL: https://build.opensuse.org/request/show/836111
OBS-URL: https://build.opensuse.org/package/show/filesystems:ceph/rook?expand=0&rev=84
- Update to v1.4.0:
* Ceph-CSI 3.0 is deployed by default
* Multi Architecture docker images are published (amd64 and arm64)
* Create/Delete beta snapshot for RBD, while support for Alpha snapshots is removed.
* Create PVCs from RBD snapshots and PVCs
* Support ROX volumes for RBD and CephFS
* The dashboard for the ceph object store will be enabled if the dashboard module is enabled.
* An admission controller enhances CRD validations (Experimental)
* The admission controller is not enabled by default.
* Support for Ceph CRDs is provided. Some validations for CephClusters are included and a framework for additional validations is in place for other CRDs.
* RGW Multisite is available through new CRDs for zones, zone groups, and realms. (Experimental)
* CephObjectStore CRD changes:
* Health displayed in the Status field
* Run health checks on the object store endpoint by creating a bucket and writing to it periodically.
* The endpoint is stored for reference in the Status field
* OSD changes:
* OSDs on PVC now support multipath and crypt device types.
* OSDs on PVC can now be encrypted by setting encrypted: true on the storageClassDeviceSet.
* OSDs can now be provisioned using Ceph's Drive Groups definitions for Ceph Octopus v15.2.5+.
* OSDs can be provisioned on the device path such as /dev/disk/by-path/pci-HHHH:HH:HH.H with colons (:)
* A new CephRBDMirror CR will configure the RBD mirroring daemons. The RBD mirror settings were previously included in the CephCluster CR.
* Multus support is improved, though still in experimental mode
* Added support for the Whereabouts IPAM
* CephCluster CRD changes:
* Converted to use the controller-runtime framework
* Added settings to configure health checks as well as pod liveness probes.
* CephBlockPool CRD has a new field called parameters which allows to set any Ceph pool property on a given pool
* OBC changes:
* Updated the lib bucket provisioner version to support multithreading
* Added support for quota, have options for object count and total size.
* Prometheus monitoring for external clusters is now possible, refer to the external cluster section
* The operator will check for the presence of the lvm2 package on the host where OSDs will run. If not available, the prepare job will fail. This will prevent issues of OSDs not restarting on node reboot.
* Added a new label ceph_daemon_type to Ceph daemon pods.
* Added a toolbox job example for running a script with Ceph commands, similar to running commands in the Rook toolbox.
OBS-URL: https://build.opensuse.org/request/show/825693
OBS-URL: https://build.opensuse.org/package/show/filesystems:ceph/rook?expand=0&rev=82
* Finalizer for OBC cleanup (#5436)
* Remove invalid MDS deactivate command during upgrade (#5278)
* Enable verbose logging for LVM commands (#5515)
* Set external creds if admin key is available (#5507)
* Fail more gracefully for an unsupported Ceph version (#5503)
* Set pg_num_min on new rgw metadata pools (#5489)
* Object store deployment failed to start on openshift (#5468)
* Relax OBC error handling and user deletion (#5465)
* Create missing secret on external cluster (#5450)
* Python script to generate needed external cluster resources (#5388)
* Docs: clarify required version of helm for upgrades (#5445)
* CSI priority class example update (#5443)
* Set test default pool size to one (#5428)
* Remove invalid verbose params from lv activate (#5438)
OBS-URL: https://build.opensuse.org/package/show/filesystems:ceph/rook?expand=0&rev=80
- Update to v1.2.7 (bsc#1168160):
* Apply the expected lower PG count for rgw metadata pools (#5091)
* Reject devices smaller than 5GiB for OSDs (#5089)
* Add extra check for filesystem to skip boot volumes for OSD configuration (#5022)
* Avoid duplication of mon pod anti-affinity (#4998)
* Update service monitor definition during upgrade (#5078)
* Resizer container fix due to misinterpretation of the cephcsi version (#5073-1)
* Set ResourceVersion for Prometheus rules (#4528)
* Upgrade doc clarification for RBAC related to the helm chart (#5054)
OBS-URL: https://build.opensuse.org/request/show/790202
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/rook?expand=0&rev=16
- Update to v1.2.7:
* Apply the expected lower PG count for rgw metadata pools (#5091)
* Reject devices smaller than 5GiB for OSDs (#5089)
* Add extra check for filesystem to skip boot volumes for OSD configuration (#5022)
* Avoid duplication of mon pod anti-affinity (#4998)
* Update service monitor definition during upgrade (#5078)
* Resizer container fix due to misinterpretation of the cephcsi version (#5073-1)
* Set ResourceVersion for Prometheus rules (#4528)
* Upgrade doc clarification for RBAC related to the helm chart (#5054)
OBS-URL: https://build.opensuse.org/request/show/790194
OBS-URL: https://build.opensuse.org/package/show/filesystems:ceph/rook?expand=0&rev=74
- Update to v1.2.2:
* Allow multiple clusters to set useAllDevices (#4692)
* Operator start all mons before checking quorum if they are all down (#4531)
* Ability to disable the crash controller (#4533)
* Document monitoring options for the cluster CR (#4698)
* Apply node topology labels to PV-backed OSDs in upgrade from v1.1 (#4616)
* Update examples to Ceph version v14.2.6 (#4653)
* Allow integration tests in minimal config to run on multiple K8s versions (#4674)
* Wrong pod name and hostname shown in alert CephMonHighNumberOfLeaderChanges (#4665)
* Set hostname properly in the CRUSH map for non-portable OSDs on PVCs (#4658)
* Update OpenShift example manifest to watch all namespaces for clusters (#4668)
* Use min_size defaults set by Ceph instead of overriding with Rook's defaults (#4638)
* CSI driver handling of upgrade from OCP 4.2 to OCP 4.3 (#4650-1)
* Add support for the k8s 1.17 failure domain labels (#4626)
* Add option to the cluster CR to continue upgrade even with unclean PGs (#4617)
* Add K8s 1.11 back to the integration tests as the minimum version (#4673)
* Fixed replication factor flag and the master addresses (#4625) (forwarded request 767768 from KGronlund)
OBS-URL: https://build.opensuse.org/request/show/767773
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/rook?expand=0&rev=12
- Update to v1.2.2:
* Allow multiple clusters to set useAllDevices (#4692)
* Operator start all mons before checking quorum if they are all down (#4531)
* Ability to disable the crash controller (#4533)
* Document monitoring options for the cluster CR (#4698)
* Apply node topology labels to PV-backed OSDs in upgrade from v1.1 (#4616)
* Update examples to Ceph version v14.2.6 (#4653)
* Allow integration tests in minimal config to run on multiple K8s versions (#4674)
* Wrong pod name and hostname shown in alert CephMonHighNumberOfLeaderChanges (#4665)
* Set hostname properly in the CRUSH map for non-portable OSDs on PVCs (#4658)
* Update OpenShift example manifest to watch all namespaces for clusters (#4668)
* Use min_size defaults set by Ceph instead of overriding with Rook's defaults (#4638)
* CSI driver handling of upgrade from OCP 4.2 to OCP 4.3 (#4650-1)
* Add support for the k8s 1.17 failure domain labels (#4626)
* Add option to the cluster CR to continue upgrade even with unclean PGs (#4617)
* Add K8s 1.11 back to the integration tests as the minimum version (#4673)
* Fixed replication factor flag and the master addresses (#4625)
OBS-URL: https://build.opensuse.org/request/show/767768
OBS-URL: https://build.opensuse.org/package/show/filesystems:ceph/rook?expand=0&rev=64
- Update rook to v1.1.7:
* Skip osd prepare job creation if osd daemon exists for the pvc (#4277)
* Stop osd process more quickly during pod shutdown to reduce IO unresponsiveness (#4328)
* Add osd anti-affinity to the example of OSDs on PVCs (#4326)
* Properly set app name on the cmdreporter (#4323)
* Ensure disruption draining state is set and checked correctly (#4319)
* Update LVM filter for OSDs on PVCs (#4312)
* Fix topology logic for disruption drains (#4221)
* Skip restorecon during ceph-volume configuration (#4260)
* Added a note around snapshot CRD cleanup (#4302)
* Storage utilization alert threshold and timing updated (#4286)
* Silence disruption errors if necessary and add missing errors (#4288)
* Create csi keys and secrets for external cluster (#4276)
* Add retry to ObjectUser creation (#4149)
OBS-URL: https://build.opensuse.org/request/show/753281
OBS-URL: https://build.opensuse.org/package/show/filesystems:ceph/rook?expand=0&rev=58
- Update rook to v1.1.6:
* Flex driver should not allow attach before detach on a different node (#3582)
* Properly set the ceph-mgr annotations (#4195)
* Only trigger an orchestration if the cluster CR changed (#4252)
* Fix setting rbdGrpcMetricsPort in the helm chart (#4202)
* Document all helm chart settings (#4202)
* Support all layers of CRUSH map with node labels (#4236)
* Skip orchestration restart on device config map update for osd on pvc (#4124)
* Deduplicate tolerations collected for the drain canary pods (#4220)
* Role bindings are missing for pod security policies (#3851)
* Continue with orchestration if a single mon pod fails to start (#4146)
* OSDs cannot call 'restorecon' when selinux is enabled (#4214)
* Use the rook image for drain canary pods (#4213)
* Allow setting of osd prepare resource limits (#4182)
* Documentation for object bucket provisioning (#3882)
OBS-URL: https://build.opensuse.org/request/show/745795
OBS-URL: https://build.opensuse.org/package/show/filesystems:ceph/rook?expand=0&rev=56
- Update rook to v1.1.4:
* OSD config overrides were ignored for some upgraded OSDs (#4161)
* Enable restoring a cluster after disaster recovery (#4021)
* Enable upgrade of OSDs configured on PVCs (#3996)
* Automatically removing OSDs requires setting: removeOSDsIfOutAndSafeToRemove(#4116)
* Rework csi keys and secrets to use minimal privileges (#4086)
* Expose OSD prepare pod resource limits (#4083)
* Minimum K8s version for running OSDs on PVCs is 1.13 (#4009)
* Add 'rgw.buckets.non-ec' to list of RGW metadataPools (#4087)
* Hide wrong error for clusterdisruption controller (#4094)
* Multiple integration test fixes to improve CI stability (#4098)
* Detect mount fstype more accurately in the flex driver (#4109)
* Do not override mgr annotations (#4110)
* Add OSDs to proper buckets in crush hierarchy with topology awareness (#4099)
* More robust removal of cluster finalizer (#4090)
* Take activeStandby into account for the CephFileSystem disruption budget (#4075)
* Update the CSI CephFS registration directory name (#4070)
* Fix incorrect Ceph CSI doc links (#4081)
* Remove decimal places for osdMemoryTargetValue monitoring setting (#4046)
* Relax pre-requisites for external cluster to allow connections to Luminous (#4025)
* Avoid nodes getting stuck in OrchestrationStatusStarting during OSD config (#3817)
* Make metrics and liveness port configurable (#4005)
* Correct system namespace for CSI driver settings during upgrade (#4040)
- Update csi-dummy-images.patch
- Update csi-template-paths.patch
- Update 0001-bsc-1152690-ceph-csi-Driver-will-fail-with-error.patch
OBS-URL: https://build.opensuse.org/request/show/745600
OBS-URL: https://build.opensuse.org/package/show/filesystems:ceph/rook?expand=0&rev=55
