Trevor Clinkenbeard
ad98d64799
Merge pull request #6473 from sfc-gh-tclinkenbeard/change-data-hall
...
Test renaming data hall on restart
2022-03-28 16:39:43 -07:00
Steve Atherton
8cf40f86e6
Merge commit '478ff1eb76bc88201b6803b8b8fb5ad9d0bcc040' into aggressive-storage-migration
2022-03-28 10:10:32 -07:00
Steve Atherton
31812d7ad3
Fix aggressive storage migration mode to behave as documented and migrates all storages at once.
2022-03-28 09:22:24 -07:00
sfc-gh-tclinkenbeard
77786f4fc6
Merge remote-tracking branch 'origin/main' into change-data-hall
2022-03-27 12:44:05 -07:00
Bharadwaj V.R
6d7a4b91c8
Create a server knob to control server version convergence threshold before SSI registration. Watch version lag in SS update loop and register when within lag limit
2022-03-27 12:37:38 -07:00
Steve Atherton
f09bdc840c
Fix undefined behavior where struct members are written to disk and restored later in a situation where they are unused but can contain random values that are not proper booleans, which ubsan complains about.
2022-03-26 19:38:59 -07:00
Steve Atherton
aa8ab494a2
Fix undefined behavior.
2022-03-26 19:37:09 -07:00
Bharadwaj V.R
f13c09eec7
Refactor SSI registration actor for error handling
2022-03-26 14:20:15 -07:00
Bharadwaj V.R
aa515524eb
Fix initialization of interface registration promise
2022-03-25 14:35:48 -07:00
Bharadwaj V.R
62b7e79482
Retract changes to apply-metadata-mutations; the change to the storagecache in commit proxy data is not required unless the get request path is to be made sensitive to the SSI state
2022-03-25 13:39:08 -07:00
Dan Lambright
827dc56c4c
Merge pull request #6623 from apple/vv
...
refactor unicast recovery
2022-03-25 16:26:24 -04:00
Dan Lambright
f867474b05
Respond to Jingyu's comments 3/24
2022-03-25 10:50:41 -04:00
Renxuan Wang
2a59c5fd4e
Workers should monitor coordinators in submitCandidacy(). ( #6655 )
...
* Workers should monitor coordinators in submitCandidacy().
* Change re-resolve delay to a knob.
2022-03-24 19:20:42 -07:00
Bharadwaj V.R
961e4ae7fd
ratekeeper and ser-des fixes
2022-03-24 17:25:07 -07:00
Trevor Clinkenbeard
cc934ff1d7
Merge pull request #6632 from sfc-gh-jslocum/dd_better_cancelling
...
Better cancelling logic that reflects whether move has actually started
2022-03-24 16:28:01 -07:00
Josh Slocum
3cd1e5599e
Cleanup
2022-03-24 17:15:11 -05:00
Josh Slocum
989dd8d7eb
Finished implementing beginVersion
2022-03-24 17:15:11 -05:00
Josh Slocum
1b1182f414
Blob Worker side of beginVersion done, added unit test
2022-03-24 17:15:11 -05:00
Ata E Husain Bohra
017709aec6
Introduce BlobCipher interface and cipher caching interface ( #6391 )
...
* Introduce BlobCipher interface and cipher caching interface
diff-3: Update the code to avoid deriving encryption key periodically.
Implement EncyrptBuf interface to limit memcpys.
Improve both unit test and simulation to better code coverage.
diff-2: Add specific error code for OpenSSL AES call failures
diff-1: Update encryption scheme to AES-256-CTR. Minor
updates to Header to capture more information.
Major changes proposed are:
1. Introduce encyrption header format.
2. Introduce a BlobCipher cipher key representation encoding
following information: baseCipher details, derived encryption cipher
details, creationTime and random salt.
3. Introduce interface to support block cipher encrytion and decrytion
operations. Encyrption populates encryption header allowing client to
persist them on-disk, this header is then read allowing decryption
on reads.
4. Introduce interface to allow in-memory caching of cipher keys. The
cache allowing mapping of "encryption domain" -> "base cipher id" ->
"derived cipher keys" (3D hash map). This cache interface will be used
by FDB processes participating in encryption to cache recently used
ciphers (performance optimization).
Testing:
1. Unit test to validate caching interface.
2. Update EncryptionOps simulation test to validate block cipher
operations.
2022-03-24 07:31:49 -07:00
Andrew Noyes
977fab2089
Make overwriteProxiesCount a function of conf keys ( #6645 )
...
* Improve assert diagnostics
* Make overwriteProxiesCount a deterministic function of conf keys
Previously, the final values for grv_proxy count, commit_proxy count,
and proxy count were derived from already-derived values from
overwriteProxiesCount. Instead, we should only look at the conf keys.
Also treat these keys as set to -1 if absent
* Add unit test
Illustrates how overwriteCommitProxy is not a function of conf keys.
Passes after change and fails before
2022-03-23 17:15:28 -07:00
He Liu
8f3dbe2575
Merge pull request #6663 from kakaiu/Add-RocksDB-Mem-Usage-Metrics
...
Add RocksDB Memory Usage Metrics
2022-03-23 14:51:23 -07:00
Zhe Wang
383b0cf0e1
add-rocksdb-memory-usage-metrics
2022-03-23 15:59:32 -04:00
Trevor Clinkenbeard
bdff100ef9
Merge pull request #6582 from sfc-gh-tclinkenbeard/global-tag-throttling2
...
Various `TagThrottler` enhancements
2022-03-23 12:54:30 -07:00
Evan Tschannen
4a085fc844
Merge pull request #6602 from apple/blob_integration
...
Blob integration
2022-03-23 12:02:43 -07:00
Dan Lambright
12e88a8ef5
Error in previous commit
2022-03-23 14:16:45 -04:00
Trevor Clinkenbeard
be57fd5b26
Fix TEST macro comment
...
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-03-23 11:07:06 -07:00
sfc-gh-tclinkenbeard
0726832e80
Merge remote-tracking branch 'origin/main' into global-tag-throttling2
2022-03-23 11:05:02 -07:00
Jingyu Zhou
0c88be0393
Refactor resolution balancing into separate files
2022-03-23 09:57:31 -07:00
Jingyu Zhou
437e7d27c6
Use trigger to start resolutionBalancing
...
Trigger when there are more than one resolvers. This also avoids the problem of
receiving multiple UpdateRecoveryDataRequests.
2022-03-23 09:57:31 -07:00
Jingyu Zhou
6e8d16538d
Remove an unused variable from MasterData
2022-03-23 09:57:31 -07:00
Jingyu Zhou
7736ea87b0
Fix code format
2022-03-23 09:57:31 -07:00
Jingyu Zhou
213e37191c
Add code coverage macro for resolution balancing
2022-03-23 09:57:31 -07:00
Jingyu Zhou
3379f1e974
Move resolutionBalancing() back to master
...
This revert the behavior done by a recent refactor on master recovery in PR #6191 .
2022-03-23 09:57:31 -07:00
neethuhaneesha
f7ec39c0ee
Merge pull request #6654 from neethuhaneesha/readAheadCompaction
...
Adding rocksdb compaction readahead size option.
2022-03-23 09:04:25 -07:00
Bharadwaj V.R
faff94ed2b
Retract changes to limit which servers are published to clients as ready for requests
2022-03-23 08:54:20 -07:00
Josh Slocum
c72197122d
bump
2022-03-23 08:07:49 -05:00
sfc-gh-tclinkenbeard
30651bf2c6
Fix order of TagInfo constructor arguments
2022-03-22 17:06:33 -07:00
Chaoguang Lin
af9deeabc2
Move the Promise<QuorumVersion> before the Future vector to be destroyed after the vector
2022-03-22 16:12:41 -07:00
Bharadwaj V.R
3a67faca7a
Re-register SSI as ready to accept requests
2022-03-22 13:41:06 -07:00
Josh Slocum
4e20da76fb
bump
2022-03-22 15:37:16 -05:00
Neethu Haneesha Bingi
790ef9ff36
Adding rocksdb compaction readahead size option.
2022-03-22 13:29:35 -07:00
Dan Lambright
f23f451cc4
Fix bug computing tlog count per log group
2022-03-22 14:12:26 -04:00
Josh Slocum
f27475e2f4
Merge branch 'main' into blob_integration
2022-03-22 11:41:58 -05:00
Steve Atherton
9f205217a4
Merge branch 'main' of https://github.com/apple/foundationdb into redwood-rare-bugs-6574
2022-03-21 21:57:57 -07:00
Josh Slocum
917c7acca5
Using desiredOldest for when change feed metadata is safe
2022-03-21 22:21:40 -05:00
Trevor Clinkenbeard
6390d93efd
Merge pull request #6646 from sfc-gh-tclinkenbeard/fix-copyright-headers
...
Update copyright header dates
2022-03-21 16:49:20 -07:00
Trevor Clinkenbeard
e04b0d7091
Merge pull request #6630 from sfc-gh-tclinkenbeard/change-rebalance-teams-signature
...
Change signature of `rebalanceTeams`
2022-03-21 16:44:16 -07:00
Josh Slocum
7b3a65676b
Fixing another change feed fetch and move race
2022-03-21 17:06:47 -05:00
Bharadwaj V.R
b4bf80c01d
Make sure SSI accepting request state is set before adding new storage servers
2022-03-21 14:47:34 -07:00
sfc-gh-tclinkenbeard
a71099471b
Update copyright header dates
2022-03-21 13:36:23 -07:00
