- update to 10.9.1:
* Fix resumable compaction incorrectly allowing resumption from a truncated
range deletion that is not well handled currently.
- update to 10.9.0:
* Added an auto-tuning feature for DB manifest file size
* Add a new option allow_trivial_move in CompactionOptions to allow
CompactFiles to perform trivial move if possible.
* To reduce risk of ODR violations or similar, ROCKSDB_USING_THREAD_STATUS
has been removed from public headers and replaced with static const bool
ThreadStatus::kEnabled.
* PosixWritableFile now repositions the seek pointer to the new end of file
after a call to Truncate.
* Updated standalone range deletion L0 file compaction behavior to avoid
compacting with any newer L0 files (which is expensive and not useful)
* Fix a bug where compaction with range deletion can persist kTypeMaxValid
in MANIFEST as file metadata.
* Fixed a bug where DB::GetSortedWalFiles() could hang when waiting for a
purge operation that found nothing to do
* Fixed a bug in MultiScan where max_sequential_skip_in_iterations could
cause the iterator to seek backward to already-unpinned blocks
* Fixed a bug for WAL_ttl_seconds > 0 use cases where the newest archived WAL
files could be incorrectly deleted when the system clock moved backwards.
* Added optimization that allowed for the asynchronous prefetching of all
data outlined in a multiscan iterator.
- update to 10.8.3:
* Add a GetColumnFamilyMetaData API variant in DB to get the SST files
intersecting a given key range.
- update to 10.8.0:
* Add kFSPrefetch to FSSupportedOps enum to allow file systems to indicate
prefetch support capability, avoiding unnecessary prefetch system calls
OBS-URL: https://build.opensuse.org/request/show/1326527
OBS-URL: https://build.opensuse.org/package/show/server:database/rocksdb?expand=0&rev=77
- update to 10.4.2:
* Fix a race condition between concurrent DB::Open sharing the same
SstFileManager instance.
- update to 10.4.1:
* RocksDB now triggers eligible compactions every 12 hours when periodic
compaction is configured. This solves a limitation of the compaction
trigger mechanism, which would only trigger compaction after specific
events like flush, compaction, or SetOptions.
* Fix a bug in BackupEngine that can crash backup due to a null
FSWritableFile passed to WritableFileWriter.
- update to 10.4.0:
* Add a new CF option memtable_avg_op_scan_flush_trigger
* Vector based memtable now supports concurrent writers
* Add new experimental
TransactionOptions::large_txn_commit_optimize_byte_threshold to enable
optimizations for large transaction commit by transaction batch data size
* Add a new option CompactionOptionsUniversal::reduce_file_locking
* Add new format_version=7 to aid experimental support of custom compression
algorithms with CompressionManager and block-based table. This format
version includes changing the format of TableProperties::compression_name.
* Public API Changes
+ Change NewExternalTableFactory to return a unique_ptr instead of
shared_ptr
+ Add an optional min file size requirement for deletion triggered
compaction.
* Fix a bug where CreateColumnFamilyWithImport() could miss the SST file for
the memtable flush it triggered. The exported CF then may not contain the
updates in the memtable when CreateColumnFamilyWithImport() is called.
* Fix iterator operations returning NotImplemented status if
disallow_memtable_writes and paranoid_memory_checks CF options are both set
OBS-URL: https://build.opensuse.org/request/show/1292404
OBS-URL: https://build.opensuse.org/package/show/server:database/rocksdb?expand=0&rev=71
- update to 10.2.1:
* Fix improper initialization of ExternalTableOptions
- update to 10.2.0:
* New Features
+ Provide histogram stats COMPACTION_PREFETCH_BYTES to measure number of
bytes for RocksDB's prefetching
+ A new API DB::GetNewestUserDefinedTimestamp is added to return the
newest user defined timestamp seen in a column family
+ Introduce API IngestWriteBatchWithIndex() for ingesting updates into DB
while bypassing memtable writes. This improves performance when writing
a large write batch to the DB.
+ Add a new CF option memtable_op_scan_flush_trigger that triggers a flush
of the memtable if an iterator's Seek()/Next() scans over a certain number
of invisible entries from the memtable.
* Public API Changes
+ AdvancedColumnFamilyOptions.max_write_buffer_number_to_maintain is
deleted. It's deprecated since introduction of a better option
max_write_buffer_size_to_maintain since RocksDB 6.5.0.
+ Deprecated API DB::MaxMemCompactionLevel().
+ Deprecated ReadOptions::ignore_range_deletions.
+ Deprecated API experimental::PromoteL0().
+ Added arbitrary string map for additional options to be overriden for
remote compactions
+ The fail_if_options_file_error option in DBOptions has been removed.
The behavior now is to always return failure in any API that fails to
persist the OPTIONS file.
* Behavior Changes
+ Make stats PREFETCH_BYTES_USEFUL, PREFETCH_HITS, PREFETCH_BYTES only
account for prefetching during user initiated scan
* Bug Fixes
OBS-URL: https://build.opensuse.org/request/show/1276278
OBS-URL: https://build.opensuse.org/package/show/server:database/rocksdb?expand=0&rev=69
- update to 10.1.3:
* Fix a bug where resurrected full_history_ts_low from a previous session
that enables UDT is used by this session that disables UDT.
- update to 10.1.2:
* Fix a bug where tail size of remote compaction output is not persisted in
primary db's manifest
- update to 10.1.0:
* New Features
+ Added a new DBOptions.calculate_sst_write_lifetime_hint_set setting
+ Add a new field num_l0_files in CompactionJobInfo
+ Added per-key-placement feature in Remote Compaction
+ Implemented API DB::GetPropertiesOfTablesByLevel
* Public API Changes
+ GetAllKeyVersions() now interprets empty slices literally as valid keys
+ DeleteFilesInRanges() now takes RangeOpt which is based on OptSlice.
The overload taking RangePtr is deprecated.
+ Add an unordered map of name/value pairs, ReadOptions::property_bag
+ Introduced CompactionServiceJobStatus::kAborted to allow handling aborted
scenario in Schedule(), Wait() or OnInstallation()
+ format_version < 2 in BlockBasedTableOptions is no longer supported for
writing new files. Support for reading such files is deprecated and might
be removed in the future.
CompressedSecondaryCacheOptions::compress_format_version == 1 is also
deprecated.
* Behavior Changes
+ ldb now returns an error if the specified --compression_type is not
supported in the build.
+ MultiGet with snapshot and ReadOptions::read_tier = kPersistedTier will
now read a consistent view across CFs
+ CreateColumnFamily() is no longer allowed on a read-only DB
OBS-URL: https://build.opensuse.org/request/show/1270226
OBS-URL: https://build.opensuse.org/package/show/server:database/rocksdb?expand=0&rev=67
- update to 9.7.3
* OPTIONS file to be loaded by remote worker is now preserved so that it does
not get purged by the primary host.
- update to 9.7.2
* Fixed a bug for surfacing write unix time:
Iterator::GetProperty("rocksdb.iterator.write-time") for non-L0 files.
- update to 9.7.1
* Fix for Several DB option settings could be lost through
GetOptionsFromString(), possibly elsewhere as well.
* Fix under counting of allocated memory in the compressed secondary cache
* Skip insertion of compressed blocks in the secondary cache if the
lowest_used_cache_tier DB option is kVolatileTier.
- update to 9.7.0
* New Features:
+ Make Cache a customizable class that can be instantiated by the object
registry.
+ Add new option prefix_seek_opt_in_only that makes iterators generally safer
+ Add a new table property "rocksdb.key.largest.seqno" which records the
largest sequence number of all keys in file.
* Behavior Changes
+ Changed the semantics of the BlobDB configuration option
blob_garbage_collection_force_threshold
+ Set write_dbid_to_manifest=true by default. This means DB ID will now be
preserved through backups, checkpoints, etc. by default. Also add
write_identity_file option
+ In FIFO compaction, compactions for changing file temperature (configured
by option file_temperature_age_thresholds) will compact one file at a time
+ Support ingesting db generated files using hard link
+ Add a new file ingestion option IngestExternalFileOptions::link_files
to hard link input files and preserve original files links after ingestion.
OBS-URL: https://build.opensuse.org/request/show/1218666
OBS-URL: https://build.opensuse.org/package/show/server:database/rocksdb?expand=0&rev=55
- update to 9.6.1:
* Fix correctness of MultiGet across column families with user timestamp.
- update to 9.6.0:
- New Features
* Best efforts recovery supports recovering to incomplete Version with a
clean seqno cut that presents a valid point in time view from the user's
perspective, if versioning history doesn't include atomic flush.
* New option BlockBasedTableOptions::decouple_partitioned_filters should
improve efficiency in serving read queries because filter and index
partitions can consistently target the configured metadata_block_size.
This option is currently opt-in.
* Introduce a new mutable CF option paranoid_memory_checks. It enables
additional validation on data integrity during reads/scanning. Currently,
skip list based memtable will validate key ordering during look up and scans.
- Public API Changes
* Add ticker stats to count file read retries due to checksum mismatch
* Adds optional installation callback function for remote compaction
- Behavior Changes
* There may be less intra-L0 compaction triggered by total L0 size being too
small. We now use compensated file size (tombstones are assigned some value
size) when calculating L0 size and reduce the threshold for L0 size limit.
This is to avoid accumulating too much data/tombstones in L0.
- Bug Fixes
* Make DestroyDB supports slow deletion when it's configured in SstFileManager.
The slow deletion is subject to the configured rate_bytes_per_sec, but not
subject to the max_trash_db_ratio.
* Fixed a bug where we set unprep_seqs_ even when WriteImpl() fails. This was
caught by stress test write fault injection in WriteImpl(). This may have
incorrectly caused iteration creation failure for unvalidated writes or
returned wrong result for WriteUnpreparedTxn::GetUnpreparedSequenceNumbers().
OBS-URL: https://build.opensuse.org/request/show/1200654
OBS-URL: https://build.opensuse.org/package/show/server:database/rocksdb?expand=0&rev=53
- update to 9.5.2:
* Fix a race condition in pessimistic transactions that could allow
multiple transactions with the same name to be registered simultaneously,
resulting in a crash or other unpredictable behavior.
* Add ticker stats to count file read retries due to checksum mismatch
- update to 9.5.1:
* Make DestroyDB supports slow deletion when it's configured in
SstFileManager. The slow deletion is subject to the configured
rate_bytes_per_sec, but not subject to the max_trash_db_ratio.
- update to 9.5.0:
* Introduced new C API function rocksdb_writebatch_iterate_cf for column
family-aware iteration over the contents of a WriteBatch
* Add support to ingest SST files generated by a DB instead of SstFileWriter.
This can be enabled with experimental option
IngestExternalFileOptions::allow_db_generated_files.
* When calculating total log size for the log_size_for_flush argument
in CreateCheckpoint API, the size of the archived log will not be
included to avoid unnecessary flush
* Fix a major bug in which an iterator using prefix filtering and SeekForPrev
might miss data when the DB is using whole_key_filtering=false and
partition_filters=true.
* Fixed a bug where OnErrorRecoveryBegin() is not called before auto
recovery starts.
* Fixed a bug where event listener reads ErrorHandler's bg_error_ member
without holding db mutex(#12803).
* Fixed a bug in handling MANIFEST write error that caused the latest valid
MANIFEST file to get deleted, resulting in the DB being unopenable.
* Fixed a race between error recovery due to manifest sync or write failure
and external SST file ingestion. Both attempt to write a new manifest file,
which causes an assertion failure.
OBS-URL: https://build.opensuse.org/request/show/1196411
OBS-URL: https://build.opensuse.org/package/show/server:database/rocksdb?expand=0&rev=51
- update to 8.11.3:
* Bug Fixes
+ Fix a bug where older data of an ingested key can be returned for read when universal compaction is used
+ Apply appropriate rate limiting and priorities in more places.
- update to 8.11.0:
* New Features
+ Add new statistics: rocksdb.sst.write.micros measures time of each write to SST file
* Public API Changes
+ Added another enumerator kVerify to enum class FileOperationType in listener.h.
Update your switch statements as needed.
+ Add CompressionOptions to the CompressedSecondaryCacheOptions structure to allow users to specify
library specific options when creating the compressed secondary cache.
+ Deprecated several options: level_compaction_dynamic_file_size, ignore_max_compaction_bytes_for_input,
+ check_flush_compaction_key_order, flush_verify_memtable_count, compaction_verify_record_count,
fail_if_options_file_error, and enforce_single_del_contracts
+ Exposed options ttl via c api.
* Behavior Changes
+ rocksdb.blobdb.blob.file.write.micros expands to also measure time writing the header and footer.
Therefore the COUNT may be higher and values may be smaller than before. For stacked BlobDB,
it no longer measures the time of explictly flushing blob file.
+ Files will be compacted to the next level if the data age exceeds periodic_compaction_seconds
except for the last level.
+ Reduced the compaction debt ratio trigger for scheduling parallel compactions
+ For leveled compaction with default compaction pri (kMinOverlappingRatio),
files marked for compaction will be prioritized over files not marked when picking a file
from a level for compaction.
* Bug Fixes
+ Fix bug in auto_readahead_size that combined with IndexType::kBinarySearchWithFirstKey + fails
or iterator lands at a wrong key
+ Fixed some cases in which DB file corruption was detected but ignored on creating a backup with BackupEngine.
OBS-URL: https://build.opensuse.org/request/show/1152968
OBS-URL: https://build.opensuse.org/package/show/server:database/rocksdb?expand=0&rev=37
- update to 8.8.1
* Bug fixes:
- Make the cache memory reservation accounting in Tiered cache
(primary and compressed secondary cache) more accurate to avoid over/under charging the secondary cache.
- Allow increasing the compressed_secondary_ratio in the Tiered cache after setting it to 0 to disable.
- update to 8.8.0
* New features:
- Introduce AttributeGroup by adding the first AttributeGroup support API, MultiGetEntity().
- Added new tickers rocksdb.fifo.{max.size|ttl}.compactions to count FIFO compactions
that drop files for different reasons
- Add an experimental offpeak duration awareness by setting DBOptions::daily_offpeak_time_utc in "HH:mm-HH:mm" format.
- Users can now change the max bytes granted in a single refill period (i.e, burst) during runtime
by SetSingleBurstBytes() for RocksDB rate limiter
* Public API Changes:
- The default value of DBOptions::fail_if_options_file_error changed from false to true.
- Add new Cache APIs GetSecondaryCacheCapacity() and GetSecondaryCachePinnedUsage()
to return the configured capacity, and cache reservation charged to the secondary cache.
* Behavior Changes:
- For non direct IO, eliminate the file system prefetching attempt for compaction read
when Options::compaction_readahead_size is 0
- During a write stop, writes now block on in-progress recovery attempts
- Deleting stale files upon recovery are delegated to SstFileManger if available so they can be rate limited.
* Bug Fixes
- Fix a bug in auto_readahead_size where first_internal_key of index blocks wasn't copied properly
resulting in corruption error when first_internal_key was used for comparison.
- Fixed a bug where compaction read under non direct IO still falls back to RocksDB internal prefetching
after file system's prefetching returns non-OK status other than Status::NotSupported()
- Add bounds check in WBWIIteratorImpl and make BaseDeltaIterator, WriteUnpreparedTxn and WritePreparedTxn
respect the upper bound and lower bound in ReadOption.
- Fixed the handling of wide-column base values in the max_successive_merges logic.
OBS-URL: https://build.opensuse.org/request/show/1128612
OBS-URL: https://build.opensuse.org/package/show/server:database/rocksdb?expand=0&rev=29
