He Liu
ca4bfb55d6
Merge branch 'main' of https://github.com/apple/foundationdb into rename-rocks-engine
2022-03-29 16:24:50 -07:00
Josh Slocum
7fc6dfa6c5
Adding useful debugging trace events
2022-03-29 14:48:28 -05:00
Jingyu Zhou
da0673ccce
Merge pull request #6705 from RenxuanW/another
...
Add proxy option to backup and restore params.
2022-03-29 11:36:13 -07:00
He Liu
dd15489605
rename ssd-rocksdb-experimental as ssd-rocksdb-v1.
2022-03-29 10:53:38 -07:00
Josh Slocum
61474d5d54
Future-proof blob granules with full file size
2022-03-29 08:06:07 -05:00
Josh Slocum
2f8e9d9de0
misc bg fixes
2022-03-29 08:05:52 -05:00
Renxuan Wang
0a332ee1c1
Add proxy option to backup and restore params.
2022-03-28 17:10:49 -07:00
Trevor Clinkenbeard
ad98d64799
Merge pull request #6473 from sfc-gh-tclinkenbeard/change-data-hall
...
Test renaming data hall on restart
2022-03-28 16:39:43 -07:00
Steve Atherton
8cf40f86e6
Merge commit '478ff1eb76bc88201b6803b8b8fb5ad9d0bcc040' into aggressive-storage-migration
2022-03-28 10:10:32 -07:00
Steve Atherton
31812d7ad3
Fix aggressive storage migration mode to behave as documented and migrates all storages at once.
2022-03-28 09:22:24 -07:00
sfc-gh-tclinkenbeard
77786f4fc6
Merge remote-tracking branch 'origin/main' into change-data-hall
2022-03-27 12:44:05 -07:00
Renxuan Wang
2a59c5fd4e
Workers should monitor coordinators in submitCandidacy(). ( #6655 )
...
* Workers should monitor coordinators in submitCandidacy().
* Change re-resolve delay to a knob.
2022-03-24 19:20:42 -07:00
Trevor Clinkenbeard
cc934ff1d7
Merge pull request #6632 from sfc-gh-jslocum/dd_better_cancelling
...
Better cancelling logic that reflects whether move has actually started
2022-03-24 16:28:01 -07:00
Josh Slocum
3cd1e5599e
Cleanup
2022-03-24 17:15:11 -05:00
Josh Slocum
989dd8d7eb
Finished implementing beginVersion
2022-03-24 17:15:11 -05:00
Josh Slocum
1b1182f414
Blob Worker side of beginVersion done, added unit test
2022-03-24 17:15:11 -05:00
Ata E Husain Bohra
017709aec6
Introduce BlobCipher interface and cipher caching interface ( #6391 )
...
* Introduce BlobCipher interface and cipher caching interface
diff-3: Update the code to avoid deriving encryption key periodically.
Implement EncyrptBuf interface to limit memcpys.
Improve both unit test and simulation to better code coverage.
diff-2: Add specific error code for OpenSSL AES call failures
diff-1: Update encryption scheme to AES-256-CTR. Minor
updates to Header to capture more information.
Major changes proposed are:
1. Introduce encyrption header format.
2. Introduce a BlobCipher cipher key representation encoding
following information: baseCipher details, derived encryption cipher
details, creationTime and random salt.
3. Introduce interface to support block cipher encrytion and decrytion
operations. Encyrption populates encryption header allowing client to
persist them on-disk, this header is then read allowing decryption
on reads.
4. Introduce interface to allow in-memory caching of cipher keys. The
cache allowing mapping of "encryption domain" -> "base cipher id" ->
"derived cipher keys" (3D hash map). This cache interface will be used
by FDB processes participating in encryption to cache recently used
ciphers (performance optimization).
Testing:
1. Unit test to validate caching interface.
2. Update EncryptionOps simulation test to validate block cipher
operations.
2022-03-24 07:31:49 -07:00
Andrew Noyes
977fab2089
Make overwriteProxiesCount a function of conf keys ( #6645 )
...
* Improve assert diagnostics
* Make overwriteProxiesCount a deterministic function of conf keys
Previously, the final values for grv_proxy count, commit_proxy count,
and proxy count were derived from already-derived values from
overwriteProxiesCount. Instead, we should only look at the conf keys.
Also treat these keys as set to -1 if absent
* Add unit test
Illustrates how overwriteCommitProxy is not a function of conf keys.
Passes after change and fails before
2022-03-23 17:15:28 -07:00
He Liu
8f3dbe2575
Merge pull request #6663 from kakaiu/Add-RocksDB-Mem-Usage-Metrics
...
Add RocksDB Memory Usage Metrics
2022-03-23 14:51:23 -07:00
Zhe Wang
383b0cf0e1
add-rocksdb-memory-usage-metrics
2022-03-23 15:59:32 -04:00
Trevor Clinkenbeard
bdff100ef9
Merge pull request #6582 from sfc-gh-tclinkenbeard/global-tag-throttling2
...
Various `TagThrottler` enhancements
2022-03-23 12:54:30 -07:00
Evan Tschannen
4a085fc844
Merge pull request #6602 from apple/blob_integration
...
Blob integration
2022-03-23 12:02:43 -07:00
Trevor Clinkenbeard
be57fd5b26
Fix TEST macro comment
...
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-03-23 11:07:06 -07:00
sfc-gh-tclinkenbeard
0726832e80
Merge remote-tracking branch 'origin/main' into global-tag-throttling2
2022-03-23 11:05:02 -07:00
Jingyu Zhou
0c88be0393
Refactor resolution balancing into separate files
2022-03-23 09:57:31 -07:00
Jingyu Zhou
437e7d27c6
Use trigger to start resolutionBalancing
...
Trigger when there are more than one resolvers. This also avoids the problem of
receiving multiple UpdateRecoveryDataRequests.
2022-03-23 09:57:31 -07:00
Jingyu Zhou
6e8d16538d
Remove an unused variable from MasterData
2022-03-23 09:57:31 -07:00
Jingyu Zhou
7736ea87b0
Fix code format
2022-03-23 09:57:31 -07:00
Jingyu Zhou
213e37191c
Add code coverage macro for resolution balancing
2022-03-23 09:57:31 -07:00
Jingyu Zhou
3379f1e974
Move resolutionBalancing() back to master
...
This revert the behavior done by a recent refactor on master recovery in PR #6191 .
2022-03-23 09:57:31 -07:00
neethuhaneesha
f7ec39c0ee
Merge pull request #6654 from neethuhaneesha/readAheadCompaction
...
Adding rocksdb compaction readahead size option.
2022-03-23 09:04:25 -07:00
Josh Slocum
c72197122d
bump
2022-03-23 08:07:49 -05:00
sfc-gh-tclinkenbeard
30651bf2c6
Fix order of TagInfo constructor arguments
2022-03-22 17:06:33 -07:00
Chaoguang Lin
af9deeabc2
Move the Promise<QuorumVersion> before the Future vector to be destroyed after the vector
2022-03-22 16:12:41 -07:00
Josh Slocum
4e20da76fb
bump
2022-03-22 15:37:16 -05:00
Neethu Haneesha Bingi
790ef9ff36
Adding rocksdb compaction readahead size option.
2022-03-22 13:29:35 -07:00
Josh Slocum
f27475e2f4
Merge branch 'main' into blob_integration
2022-03-22 11:41:58 -05:00
Josh Slocum
917c7acca5
Using desiredOldest for when change feed metadata is safe
2022-03-21 22:21:40 -05:00
Trevor Clinkenbeard
6390d93efd
Merge pull request #6646 from sfc-gh-tclinkenbeard/fix-copyright-headers
...
Update copyright header dates
2022-03-21 16:49:20 -07:00
Trevor Clinkenbeard
e04b0d7091
Merge pull request #6630 from sfc-gh-tclinkenbeard/change-rebalance-teams-signature
...
Change signature of `rebalanceTeams`
2022-03-21 16:44:16 -07:00
Josh Slocum
7b3a65676b
Fixing another change feed fetch and move race
2022-03-21 17:06:47 -05:00
sfc-gh-tclinkenbeard
a71099471b
Update copyright header dates
2022-03-21 13:36:23 -07:00
Jingyu Zhou
e47f12d985
Merge pull request #6641 from sfc-gh-xwang/main
...
remove some fixme in workload status fetcher
2022-03-21 13:26:09 -07:00
Jingyu Zhou
fe7892c2d6
Merge pull request #6579 from sfc-gh-tclinkenbeard/ddteamcollection-boolean-param
...
Improve type safety of `DDTeamCollection` boolean parameters
2022-03-21 13:18:32 -07:00
Jingyu Zhou
fb046c0665
Merge pull request #6615 from sfc-gh-tclinkenbeard/add-missing-copyright-headers
...
Add missing copyright headers
2022-03-21 13:17:15 -07:00
Jingyu Zhou
44bce12216
Merge pull request #6635 from sfc-gh-etschannen/fix-txn-recovery
...
Do not reply to a txnStateRequest for the final sequence unless we have processes the txnStateStore
2022-03-21 13:13:44 -07:00
Xiaoxi Wang
66d02d1ebf
remove some fixme in workload status fetcher
2022-03-21 11:16:46 -07:00
Josh Slocum
58c0f3d772
Handling change feed create mutation getting rolled back properly
2022-03-21 12:46:59 -05:00
A.J. Beamon
9d8884adb0
Merge pull request #6636 from sfc-gh-ajbeamon/fix-tenant-test-prefix
...
Limit tenant subspace to the normal key space
2022-03-21 10:08:32 -07:00
Josh Slocum
f4c2a789c6
Explicitly cancelling file updater future
2022-03-21 11:47:40 -05:00
