mirror of
https://github.com/apple/foundationdb.git
synced 2025-05-28 10:52:03 +08:00
Consolidated 7.0 Release Notes.
This commit is contained in:
negoyal
parent
09b0c1c419
commit
eec691b6d3
@ -9,9 +9,23 @@ Release Notes
|
||||
|
||||
Features
|
||||
--------
|
||||
* Replaced committed version broadcast through proxy with centralizing live committed versions into master. `(PR #3307) <https://github.com/apple/foundationdb/pull/3307>`_
|
||||
* Added a new API in all bindings that can be used to get a list of split points that will split the given range into (roughly) equally sized chunks. `(PR #3394) <https://github.com/apple/foundationdb/pull/3394>`_
|
||||
* Introduced a new role called GRV proxy specialized for serving GRV requests to decrease GRV tail latency since we prioritize commit paths over GRV in the current proxy. The original proxy is renamed to Commit proxy. `(PR #3549) <https://github.com/apple/foundationdb/pull/3549>`_ `(PR #3772) <https://github.com/apple/foundationdb/pull/3772>`_
|
||||
* Added support for writing backup files directly to Azure blob storage. This is not yet performance tested on large-scale clusters. `(PR #3961) <https://github.com/apple/foundationdb/pull/3961>`_
|
||||
* Tag-based throttling now also takes the write path into account. `(PR #3512) <https://github.com/apple/foundationdb/pull/3512>`_
|
||||
* Added the ability to ratekeeper to throttle certain types of tags based on write hot spots in addition to read hot spots. `(PR #3571) <https://github.com/apple/foundationdb/pull/3571>`_
|
||||
* Users now have the option to make ratekeeper recommend which transaction tags should be throttled, but not actually throttle them using fdbcli . `(PR #3669) <https://github.com/apple/foundationdb/pull/3669>`_
|
||||
* Added a new `--build_flags option` to binaries to print build information. `(PR #3769)<https://github.com/apple/foundationdb/pull/3769>`_
|
||||
* Added `--incremental` option to backup and restore that allows specification of only recording mutation log files and not range files. Incremental restore also allows restoring to a non-empty destination database. `(PR #3676) <https://github.com/apple/foundationdb/pull/3676>`_
|
||||
* Added request tracing framework. `(PR #3329) <https://github.com/apple/foundationdb/pull/3329>`_ `(PR #3700) <https://github.com/apple/foundationdb/pull/3700>`_ `(PR #4048) <https://github.com/apple/foundationdb/pull/4048>`_ `(PR #4125) <https://github.com/apple/foundationdb/pull/4125>`_ `(PR #4204) <https://github.com/apple/foundationdb/pull/4204>`_
|
||||
* Added the `global configuration framework <https://github.com/apple/foundationdb/wiki/Global-Configuration-Framework>`, an eventually consistent configuration mechanism to efficiently make runtime changes to all clients and servers. `(PR #4330) <https://github.com/apple/foundationdb/pull/4330>`_
|
||||
* Added the feature to monitor and manage an fdb cluster via read/write specific special keys through transactions. `(PR #3455) <https://github.com/apple/foundationdb/pull/3455>`_ `(PR #3945) <<https://github.com/apple/foundationdb/pull/3945>`_ `(PR #4255) <https://github.com/apple/foundationdb/pull/4255>`_ `(PR #4354) <https://github.com/apple/foundationdb/pull/4354>`_ `(PR #4582) <https://github.com/apple/foundationdb/pull/4582>`_
|
||||
* Added TLS support to fdbdecode for decoding mutation log files stored in blobs. `(PR #4611) <https://github.com/apple/foundationdb/pull/4611>`_
|
||||
* Added ``initial_snapshot_interval`` to fdbbackup that can specify the duration of the first inconsistent snapshot written to the backup. `(PR #4620) <https://github.com/apple/foundationdb/pull/4620>`_
|
||||
* Added ``inconsistent_snapshot_only`` to fdbbackup that ignores mutation log files and only uses range files during the restore to speedup the process. `(PR #4704) <https://github.com/apple/foundationdb/pull/4704>`_
|
||||
* Added the Testing Storage Server (TSS), which allows FoundationDB to run an "untrusted" storage engine with identical workload to the current storage engine, with zero impact on durability or correctness, and minimal impact on performance. `(PR #4556) <https://github.com/apple/foundationdb/pull/4556>`_ `(PR #4892) <https://github.com/apple/foundationdb/pull/4892>`_ `(PR #4895) <https://github.com/apple/foundationdb/pull/4895>`_ `(PR #4934) <https://github.com/apple/foundationdb/pull/4934>`_ `(PR #4949) <https://github.com/apple/foundationdb/pull/4949>`_ `(PR #4965) <https://github.com/apple/foundationdb/pull/4965>`_
|
||||
* Added perpetual storage wiggle that supports less impactful B-trees recreation and data migration. These will also be used for deploying the Testing Storage Server which compares 2 storage engines' results. `(PR #4838) <https://github.com/apple/foundationdb/pull/4838>`_ `(PR #4860) <https://github.com/apple/foundationdb/pull/4860`_ `(PR #5009) <https://github.com/apple/foundationdb/pull/5009>`_
|
||||
|
||||
Performance
|
||||
-----------
|
||||
@ -19,29 +33,41 @@ Performance
|
||||
* Increased performance of dr_agent when copying the mutation log. The ``COPY_LOG_BLOCK_SIZE``, ``COPY_LOG_BLOCKS_PER_TASK``, ``COPY_LOG_PREFETCH_BLOCKS``, ``COPY_LOG_READ_AHEAD_BYTES`` and ``COPY_LOG_TASK_DURATION_NANOS`` knobs can be set. `(PR #3436) <https://github.com/apple/foundationdb/pull/3436>`_
|
||||
* Added multiple new microbenchmarks for PromiseStream, Reference, IRandom, and timer, as well as support for benchmarking actors. `(PR #3590) <https://github.com/apple/foundationdb/pull/3590>`_
|
||||
* Use xxhash3 for SQLite page checksums. `(PR #4075) <https://github.com/apple/foundationdb/pull/4075>`_
|
||||
* Reduced the number of connections required by the multi-version client when loading external clients. When connecting to 7.0 clusters, only one connection with version 6.2 or larger will be used. With older clusters, at most two connections with version 6.2 or larger will be used. Clients older than version 6.2 will continue to create an additional connection each. `(PR #4667) <https://github.com/apple/foundationdb/pull/4667>`_
|
||||
* fdbserver now uses jemalloc on Linux instead of the system malloc. `(PR #4222) <https://github.com/apple/foundationdb/pull/4222>`_
|
||||
* Watches have been optimized and are now significantly cheaper. `(PR #4266) <https://github.com/apple/foundationdb/pull/4266>`_ `(PR #4382 ) <https://github.com/apple/foundationdb/pull/4382>`_
|
||||
* The Coro library has been replaced with boost::coroutine2. `(PR #4242) <https://github.com/apple/foundationdb/pull/4242>`_
|
||||
* Reduce CPU overhead of load balancing on client processes. `(PR #4561) <https://github.com/apple/foundationdb/pull/4561>`_
|
||||
* Used the restored key range to filter out files for faster restore. `(PR #4568) <https://github.com/apple/foundationdb/pull/4568>`_
|
||||
* Reduced the number of connections required by the multi-version client when loading external clients. When connecting to 7.0 clusters, only one connection with version 6.2 or larger will be used. With older clusters, at most two connections with version 6.2 or larger will be used. Clients older than version 6.2 will continue to create an additional connection each. `(PR #4667) <https://github.com/apple/foundationdb/pull/4667>`_
|
||||
|
||||
Reliability
|
||||
-----------
|
||||
* Improved worker recruitment logic to avoid unnecessary recoveries when processes are added or removed from a cluster. `(PR #4695) <https://github.com/apple/foundationdb/pull/4695>`_ `(PR #4631) <https://github.com/apple/foundationdb/pull/4631>`_ `(PR #4509) <https://github.com/apple/foundationdb/pull/4509>`_
|
||||
* Backup agents now pause themselves upon a successful snapshot recovery to avoid unintentional data corruption. Operators should manually abort backup agents and clear the backup agent keyspace to avoid using the old cluster's backup configuration. `(PR #4027) <https://github.com/apple/foundationdb/pull/4027>`_
|
||||
* Log class processes are prioritized above transaction class proceses for becoming tlogs. `(PR #4509) <https://github.com/apple/foundationdb/pull/4509>`_
|
||||
* Improved worker recruitment logic to avoid unnecessary recoveries when processes are added or removed from a cluster. `(PR #4695) <https://github.com/apple/foundationdb/pull/4695>`_ `(PR #4631) <https://github.com/apple/foundationdb/pull/4631>`_ `(PR #4509) <https://github.com/apple/foundationdb/pull/4509>`_
|
||||
|
||||
Fixes
|
||||
-----
|
||||
* Fixed a rare crash on the cluster controller when using multi-region configurations. `(PR #4547) <https://github.com/apple/foundationdb/pull/4547>`_
|
||||
* List files asynchronously so many backup files on a slow disk won't cause the backup agent to lose its lease. `(PR #3094) <https://github.com/apple/foundationdb/pull/3094>`_
|
||||
* Unknown endpoint has been tracked in a wrong way and therefore showed up too frequently in our statistics. `(PR #4473) <https://github.com/apple/foundationdb/pull/4473>`_
|
||||
* Using the ``exclude failed`` command could leave the data distributor in a state where it cannot complete relocations. `(PR #4495) <https://github.com/apple/foundationdb/pull/4495>`_
|
||||
* Fixed a rare crash on the cluster controller when using multi-region configurations. `(PR #4547) <https://github.com/apple/foundationdb/pull/4547>`_
|
||||
* Fixed a memory corruption bug in the data distributor. `(PR #4535) <https://github.com/apple/foundationdb/pull/4535>`_
|
||||
* Fixed a rare crash that could happen on the sequencer during recovery. `(PR #4548) <https://github.com/apple/foundationdb/pull/4548>`_
|
||||
* Added a new pre-backup action when creating a backup. Backups can now either verify the range data is being saved to is empty before the backup begins (current behavior) or clear the range where data is being saved to. Fixes a `restore_destination_not_empty` failure after a backup retry due to `commit_unknown_failure`. `(PR #4595) <https://github.com/apple/foundationdb/pull/4595>`_
|
||||
* When configured with ``usable_regions=2``, a cluster would not fail over to a region which contained only storage class processes. `(PR #4599) <https://github.com/apple/foundationdb/pull/4599>`_
|
||||
|
||||
Status
|
||||
------
|
||||
* Added ``cluster.qos.throttled_tags`` and ``cluster.processes.*.roles.busiest_[read|write]_tag`` to report statistics on throttled tags and the busiest read or write transaction tags on each storage server. `(PR #3669) <https://github.com/apple/foundationdb/pull/3669>`_ `(PR #3696) <https://github.com/apple/foundationdb/pull/3696>`_
|
||||
* Added ``seconds_since_last_recovered`` to the ``cluster.recovery_state`` section to report how long it has been since the cluster recovered to the point where it is able to accept requests. `(PR #3759) <https://github.com/apple/foundationdb/pull/3759>`_
|
||||
* Added limiting metrics (limiting_storage_durability_lag and limiting_storage_queue) to health metrics. `(PR #4067) <https://github.com/apple/foundationdb/pull/4067>`_
|
||||
* ``min_replicas_remaining`` is now populated for all regions, thus giving a clear picture of the data replicas that exist in the database. `(PR 4515) <https://github.com/apple/foundationdb/pull/4515>`_
|
||||
* Added detailed metrics for batched transactions.`(PR #4540) <https://github.com/apple/foundationdb/pull/4540>`_
|
||||
* Added ``commit_batching_window_size`` to the proxy roles section of status to record statistics about commit batching window size on each proxy. `(PR #4735) <https://github.com/apple/foundationdb/pull/4735>`_
|
||||
* Added ``cluster.bounce_impact`` section to status to report if there will be any extra effects when bouncing the cluster, and if so, the reason for those effects. `(PR #4770) <https://github.com/apple/foundationdb/pull/4770>`_
|
||||
* Added ``fetched_versions`` to the storage metrics section of status to report how fast a storage server is catching up in versions. `(PR #4770) <https://github.com/apple/foundationdb/pull/4770>`_
|
||||
* Added ``fetches_from_logs`` to the storage metrics section of status to report how frequently a storage server fetches updates from transaction logs. `(PR #4770) <https://github.com/apple/foundationdb/pull/4770>`_
|
||||
* Added ``seconds_since_last_recovered`` to the ``cluster.recovery_state`` section to report how long it has been since the cluster recovered to the point where it is able to accept requests. `(PR #3759) <https://github.com/apple/foundationdb/pull/3759>`_
|
||||
|
||||
Bindings
|
||||
--------
|
||||
@ -54,6 +80,7 @@ Bindings
|
||||
|
||||
Other Changes
|
||||
-------------
|
||||
* Added rte_memcpy from DPDK for default usage. `(PR #3089) <https://github.com/apple/foundationdb/pull/3089/files>`_
|
||||
* When ``fdbmonitor`` dies, all of its child processes are now killed. `(PR #3841) <https://github.com/apple/foundationdb/pull/3841>`_
|
||||
* The ``foundationdb`` service installed by the RPM packages will now automatically restart ``fdbmonitor`` after 60 seconds when it fails. `(PR #3841) <https://github.com/apple/foundationdb/pull/3841>`_
|
||||
* Capture output of forked snapshot processes in trace events. `(PR #4254) <https://github.com/apple/foundationdb/pull/4254/files>`_
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user