* OPTIONS file to be loaded by remote worker is now preserved so that it does
not get purged by the primary host.
- update to 9.7.2
* Fixed a bug for surfacing write unix time:
Iterator::GetProperty("rocksdb.iterator.write-time") for non-L0 files.
- update to 9.7.1
* Fix for Several DB option settings could be lost through
GetOptionsFromString(), possibly elsewhere as well.
* Fix under counting of allocated memory in the compressed secondary cache
* Skip insertion of compressed blocks in the secondary cache if the
lowest_used_cache_tier DB option is kVolatileTier.
- update to 9.7.0
* New Features:
+ Make Cache a customizable class that can be instantiated by the object
registry.
+ Add new option prefix_seek_opt_in_only that makes iterators generally safer
+ Add a new table property "rocksdb.key.largest.seqno" which records the
largest sequence number of all keys in file.
* Behavior Changes
+ Changed the semantics of the BlobDB configuration option
blob_garbage_collection_force_threshold
+ Set write_dbid_to_manifest=true by default. This means DB ID will now be
preserved through backups, checkpoints, etc. by default. Also add
write_identity_file option
+ In FIFO compaction, compactions for changing file temperature (configured
by option file_temperature_age_thresholds) will compact one file at a time
+ Support ingesting db generated files using hard link
+ Add a new file ingestion option IngestExternalFileOptions::link_files
to hard link input files and preserve original files links after ingestion.
OBS-URL: https://build.opensuse.org/package/show/server:database/rocksdb?expand=0&rev=55
* Fix correctness of MultiGet across column families with user timestamp.
- update to 9.6.0:
- New Features
* Best efforts recovery supports recovering to incomplete Version with a
clean seqno cut that presents a valid point in time view from the user's
perspective, if versioning history doesn't include atomic flush.
* New option BlockBasedTableOptions::decouple_partitioned_filters should
improve efficiency in serving read queries because filter and index
partitions can consistently target the configured metadata_block_size.
This option is currently opt-in.
* Introduce a new mutable CF option paranoid_memory_checks. It enables
additional validation on data integrity during reads/scanning. Currently,
skip list based memtable will validate key ordering during look up and scans.
- Public API Changes
* Add ticker stats to count file read retries due to checksum mismatch
* Adds optional installation callback function for remote compaction
- Behavior Changes
* There may be less intra-L0 compaction triggered by total L0 size being too
small. We now use compensated file size (tombstones are assigned some value
size) when calculating L0 size and reduce the threshold for L0 size limit.
This is to avoid accumulating too much data/tombstones in L0.
- Bug Fixes
* Make DestroyDB supports slow deletion when it's configured in SstFileManager.
The slow deletion is subject to the configured rate_bytes_per_sec, but not
subject to the max_trash_db_ratio.
* Fixed a bug where we set unprep_seqs_ even when WriteImpl() fails. This was
caught by stress test write fault injection in WriteImpl(). This may have
incorrectly caused iteration creation failure for unvalidated writes or
returned wrong result for WriteUnpreparedTxn::GetUnpreparedSequenceNumbers().
OBS-URL: https://build.opensuse.org/package/show/server:database/rocksdb?expand=0&rev=53
* Fix a race condition in pessimistic transactions that could allow
multiple transactions with the same name to be registered simultaneously,
resulting in a crash or other unpredictable behavior.
* Add ticker stats to count file read retries due to checksum mismatch
- update to 9.5.1:
* Make DestroyDB supports slow deletion when it's configured in
SstFileManager. The slow deletion is subject to the configured
rate_bytes_per_sec, but not subject to the max_trash_db_ratio.
- update to 9.5.0:
* Introduced new C API function rocksdb_writebatch_iterate_cf for column
family-aware iteration over the contents of a WriteBatch
* Add support to ingest SST files generated by a DB instead of SstFileWriter.
This can be enabled with experimental option
IngestExternalFileOptions::allow_db_generated_files.
* When calculating total log size for the log_size_for_flush argument
in CreateCheckpoint API, the size of the archived log will not be
included to avoid unnecessary flush
* Fix a major bug in which an iterator using prefix filtering and SeekForPrev
might miss data when the DB is using whole_key_filtering=false and
partition_filters=true.
* Fixed a bug where OnErrorRecoveryBegin() is not called before auto
recovery starts.
* Fixed a bug where event listener reads ErrorHandler's bg_error_ member
without holding db mutex(#12803).
* Fixed a bug in handling MANIFEST write error that caused the latest valid
MANIFEST file to get deleted, resulting in the DB being unopenable.
* Fixed a race between error recovery due to manifest sync or write failure
and external SST file ingestion. Both attempt to write a new manifest file,
which causes an assertion failure.
OBS-URL: https://build.opensuse.org/package/show/server:database/rocksdb?expand=0&rev=51
- update to 8.11.3:
* Bug Fixes
+ Fix a bug where older data of an ingested key can be returned for read when universal compaction is used
+ Apply appropriate rate limiting and priorities in more places.
- update to 8.11.0:
* New Features
+ Add new statistics: rocksdb.sst.write.micros measures time of each write to SST file
* Public API Changes
+ Added another enumerator kVerify to enum class FileOperationType in listener.h.
Update your switch statements as needed.
+ Add CompressionOptions to the CompressedSecondaryCacheOptions structure to allow users to specify
library specific options when creating the compressed secondary cache.
+ Deprecated several options: level_compaction_dynamic_file_size, ignore_max_compaction_bytes_for_input,
+ check_flush_compaction_key_order, flush_verify_memtable_count, compaction_verify_record_count,
fail_if_options_file_error, and enforce_single_del_contracts
+ Exposed options ttl via c api.
* Behavior Changes
+ rocksdb.blobdb.blob.file.write.micros expands to also measure time writing the header and footer.
Therefore the COUNT may be higher and values may be smaller than before. For stacked BlobDB,
it no longer measures the time of explictly flushing blob file.
+ Files will be compacted to the next level if the data age exceeds periodic_compaction_seconds
except for the last level.
+ Reduced the compaction debt ratio trigger for scheduling parallel compactions
+ For leveled compaction with default compaction pri (kMinOverlappingRatio),
files marked for compaction will be prioritized over files not marked when picking a file
from a level for compaction.
* Bug Fixes
+ Fix bug in auto_readahead_size that combined with IndexType::kBinarySearchWithFirstKey + fails
or iterator lands at a wrong key
+ Fixed some cases in which DB file corruption was detected but ignored on creating a backup with BackupEngine.
OBS-URL: https://build.opensuse.org/request/show/1152968
OBS-URL: https://build.opensuse.org/package/show/server:database/rocksdb?expand=0&rev=37
- update to 8.8.1
* Bug fixes:
- Make the cache memory reservation accounting in Tiered cache
(primary and compressed secondary cache) more accurate to avoid over/under charging the secondary cache.
- Allow increasing the compressed_secondary_ratio in the Tiered cache after setting it to 0 to disable.
- update to 8.8.0
* New features:
- Introduce AttributeGroup by adding the first AttributeGroup support API, MultiGetEntity().
- Added new tickers rocksdb.fifo.{max.size|ttl}.compactions to count FIFO compactions
that drop files for different reasons
- Add an experimental offpeak duration awareness by setting DBOptions::daily_offpeak_time_utc in "HH:mm-HH:mm" format.
- Users can now change the max bytes granted in a single refill period (i.e, burst) during runtime
by SetSingleBurstBytes() for RocksDB rate limiter
* Public API Changes:
- The default value of DBOptions::fail_if_options_file_error changed from false to true.
- Add new Cache APIs GetSecondaryCacheCapacity() and GetSecondaryCachePinnedUsage()
to return the configured capacity, and cache reservation charged to the secondary cache.
* Behavior Changes:
- For non direct IO, eliminate the file system prefetching attempt for compaction read
when Options::compaction_readahead_size is 0
- During a write stop, writes now block on in-progress recovery attempts
- Deleting stale files upon recovery are delegated to SstFileManger if available so they can be rate limited.
* Bug Fixes
- Fix a bug in auto_readahead_size where first_internal_key of index blocks wasn't copied properly
resulting in corruption error when first_internal_key was used for comparison.
- Fixed a bug where compaction read under non direct IO still falls back to RocksDB internal prefetching
after file system's prefetching returns non-OK status other than Status::NotSupported()
- Add bounds check in WBWIIteratorImpl and make BaseDeltaIterator, WriteUnpreparedTxn and WritePreparedTxn
respect the upper bound and lower bound in ReadOption.
- Fixed the handling of wide-column base values in the max_successive_merges logic.
OBS-URL: https://build.opensuse.org/request/show/1128612
OBS-URL: https://build.opensuse.org/package/show/server:database/rocksdb?expand=0&rev=29
- update to 8.6.7
* Fixed a bug where compaction read under non direct IO still falls back to RocksDB internal prefetching after file system's prefetching returns non-OK status other than Status::NotSupported()
* For non direct IO, eliminate the file system prefetching attempt for compaction read when Options::compaction_readahead_size is 0
- update to 8.6.6
* Fix a bug with atomic_flush=true that can cause DB to stuck after a flush fails
* Fix a bug where RocksDB (with atomic_flush=false) can delete output SST files of pending flushes when a previous concurrent flush fails
* When the compressed secondary cache capacity is reduced to 0, it should be completely disabled. Before this fix, inserts and lookups would still go to the backing LRUCache
- update to 8.6.5
* Fixed a bug where rocksdb.file.read.verify.file.checksums.micros is not populated.
- update to 8.6.4
* Public API change: Add a column family option default_temperature that is used for file reading accounting purpose, such as io statistics, for files that don't have an explicitly set temperature.
- update to 8.6.3
* Fix a bug where if there is an error reading from offset 0 of a file from L1+ and that the file is not the first file in the sorted run, data can be lost in compaction and read/scan can return incorrect results.
* Fix a bug where iterator may return incorrect result for DeleteRange() users if there was an error reading from a file.
- update to 8.6.2
* Fix for async_io where during seek, when reading a block for seeking a target key in a file without any readahead, the iterator aligned the read on a page boundary and reading more than necessary. This increased the storage read bandwidth usage.
- update to 8.6.1
* Options::compaction_readahead_size 's default value is changed from 0 to 2MB.
* Compaction read performance will regress when Options::compaction_readahead_size is explicitly set to 0
- update to 8.6.0
* Added enhanced data integrity checking on SST files with new format_version=6.
* Add a new feature to trim readahead_size during scans upto upper_bound when iterate_upper_bound is specified.
* RocksDB will compare the number of input keys to the number of keys processed after each compaction.
* Add a CF option bottommost_file_compaction_delay to allow specifying the delay of bottommost level single-file compactions.
* Add support to allow enabling / disabling user-defined timestamps feature for an existing column family in combination with the in-Memtable only feature.
* Implement a new admission policy for the compressed secondary cache that admits blocks evicted from the primary cache with the hit bit set.
* Add a column family option memtable_max_range_deletions that limits the number of range deletions in a memtable.
* Add PutEntity API in sst_file_writer
* Add timeout in microsecond option to WaitForCompactOptions
* New statistics rocksdb.file.read.{get|multiget|db.iterator|verify.checksum|verify.file.checksums}.micros measure read time of block-based SST tables or blob files during db open, Get(), MultiGet(), using db iterator, VerifyFileChecksums() and VerifyChecksum(). They require stats level greater than StatsLevel::kExceptDetailedTimers.
OBS-URL: https://build.opensuse.org/request/show/1116481
OBS-URL: https://build.opensuse.org/package/show/server:database/rocksdb?expand=0&rev=24
- update to 8.5.3
* Fixed a race condition in GenericRateLimiter that could cause it to stop granting requests
- update to 8.5.2
* Fix a bug where iterator may return incorrect result for DeleteRange() users if there was an error reading from a file.
- update to 8.5.1
* Fix a bug where if there is an error reading from offset 0 of a file from L1+ and that the file is not the first file in the sorted run, data can be lost in compaction and read/scan can return incorrect results.
- update to 8.5.0
* Public API Changes:
Removed recently added APIs GeneralCache and MakeSharedGeneralCache() as our plan changed to stop exposing a general-purpose cache interface. The old forms of these APIs, Cache and NewLRUCache(), are still available, although general-purpose caching support will be dropped eventually.
* Behavior Changes
Option periodic_compaction_seconds no longer supports FIFO compaction: setting it has no effect on FIFO compactions. FIFO compaction users should only set option ttl instead.
Move prefetching responsibility to page cache for compaction read for non directIO use case
* Performance Improvements
In case of direct_io, if buffer passed by callee is already aligned, RandomAccessFileRead::Read will avoid realloacting a new buffer, reducing memcpy and use already passed aligned buffer.
Small efficiency improvement to HyperClockCache by reducing chance of compiler-generated heap allocations
* Bug Fixes
Fix use_after_free bug in async_io MultiReads when underlying FS enabled kFSBuffer. kFSBuffer is when underlying FS pass their own buffer instead of using RocksDB scratch in FSReadRequest. Right now it's an experimental feature.
Fix a bug in FileTTLBooster that can cause users with a large number of levels (more than 65) to see errors like "runtime error: shift exponent .. is too large.."
- see more on HISTORY.md (https://github.com/facebook/rocksdb/blob/main/HISTORY.md)
OBS-URL: https://build.opensuse.org/request/show/1109202
OBS-URL: https://build.opensuse.org/package/show/server:database/rocksdb?expand=0&rev=20
- update to 8.3.2
* Bug Fixes:
Reduced cases of illegally using Env::Default() during static destruction by never destroying the internal PosixEnv itself
- update to 8.3.1
* Performance Improvements:
Fixed higher read QPS during DB::Open() reading files created prior to #11406,
- update to 8.3.0
* New Features:
Introduced a new option block_protection_bytes_per_key, which can be used to enable per key-value integrity protection for in-memory blocks in block cache
Improve the operational safety of publishing a DB or SST files to many hosts by using different block cache hash seeds on different hosts.
Introduced a new option CompactionOptionsFIFO::file_temperature_age_thresholds that allows FIFO compaction to compact files to different temperatures based on key age
Added a new ticker stat to count how many times RocksDB detected a corruption while verifying a block checksum: BLOCK_CHECKSUM_MISMATCH_COUNT.
New statistics rocksdb.file.read.db.open.micros that measures read time of block-based SST tables or blob files during db open.
New statistics tickers for various iterator seek behaviors and relevant filtering, as *_LEVEL_SEEK_*. (#11460)
* Public API Changes:
EXPERIMENTAL: Add new API DB::ClipColumnFamily to clip the key in CF to a certain range. It will physically deletes all keys outside the range including tombstones.
Add MakeSharedCache() construction functions to various cache Options objects, and deprecated the NewWhateverCache() functions with long parameter lists.
Changed the meaning of various Bloom filter stats (prefix vs. whole key), with iterator-related filtering only being tracked in the new *_LEVEL_SEEK_*. stats. (#11460)
* Behavior changes:
For x86, CPU features are no longer detected at runtime nor in build scripts, but in source code using common preprocessor defines.
This will likely unlock some small performance improvements on some newer hardware, but could hurt performance of the kCRC32c checksum,
which is no longer the default, on some "portable" builds. See PR #11419 for details.
* Bug Fixes:
Delete an empty WAL file on DB open if the log number is less than the min log number to keep
Delete temp OPTIONS file on DB open if there is a failure to write it out or rename it
* Performance Improvements:
Improved the I/O efficiency of prefetching SST metadata by recording more information in the DB manifest.
OBS-URL: https://build.opensuse.org/request/show/1101081
OBS-URL: https://build.opensuse.org/package/show/server:database/rocksdb?expand=0&rev=18
