Josh Slocum
aef5130da2
adding system priority option to getDatabaseConfiguration, and several debugging improvements ( #9864 )
2023-04-06 15:08:40 -05:00
Hui Liu
396f89a3f4
Cleanup stale disk files for double recruitment of storage server ( #9794 )
2023-04-06 12:13:59 -07:00
Ata E Husain Bohra
ecc6d5a712
EaR: Fix BlobCipher cache handling for cipher needs refresh and/or expired ( #9845 )
...
* EaR: Fix BlobCipher cache handling for cipher needs refresh and/or expired
Description
Patch proposes BlobCipher cache bug related to handling of cipherKeys
that either 'needsRefresh' and/or 'expired'
Also, adds a unit-test to cover the following usecase:
1. Test refreshAt and expireAt properties of the cipherKey
2. Validate corresponding Counter value increments
Testing
Extend /blobCipher unitest tests
2023-04-06 11:43:10 -07:00
Nim Wijetunga
c780d706d1
Remove EKP Interface from ServerDBInfo ( #9909 )
...
Remove ekp interface from ServerDBInfo
2023-04-06 11:35:47 -07:00
Hui Liu
711e040627
RestoreConfig - use restoreRangeSet to replace restoreRanges ( #9912 )
2023-04-06 11:16:05 -07:00
Ata E Husain Bohra
625eb00d36
Relax EKP failure detection time for simulated runs ( #9908 )
...
Description
EKP failure detection time is choosen to be fairly low given EKP
availability could impact cluster availability. However, in
simulation runs, for various reasons such a low-latency failure
detection could cause EKP recruitment to be done in a loop.
Patch relaxes the failure detection time for simulation runs
Testing
2023-04-05 18:12:25 -07:00
Nim Wijetunga
6e4e6ab2f4
Revert "Revert "Refactor GetEncryptCipherKeys ( #9600 )"" ( #9903 )
...
* Revert "Revert "Refactor GetEncryptCipherKeys (#9600 )" (#9708 )"
2023-04-05 10:03:48 -07:00
Jingyu Zhou
6b37163473
Merge pull request #9902 from sfc-gh-clin/fix-versionepoch-test
...
Add wait at the end of the versionepoch fdbcli test which triggers recovery
2023-04-05 08:59:13 -07:00
A.J. Beamon
fcf34eb1f2
Merge pull request #9899 from sfc-gh-ajbeamon/update-tenant-group-index
...
Update the tenant group index to include the tenant name
2023-04-05 08:52:18 -07:00
Chaoguang Lin
b9935ef6b4
Add wait at the end of versionepoch which triggers recovery; add start&end logging of each command test
2023-04-04 17:05:47 -07:00
A.J. Beamon
6023ee866a
Fix format
2023-04-04 16:10:56 -07:00
A.J. Beamon
974376b484
Use originalEntry rather than updatedTenantEntry to access the tenant name for the tenant index entry being erased.
2023-04-04 16:02:26 -07:00
A.J. Beamon
e27908556a
Update the tenant group index to be a tuple of (tenant group name, tenant name, tenant ID)
2023-04-04 14:46:15 -07:00
Jingyu Zhou
c5b8908828
Update 7.1.30 release notes ( #9887 )
2023-04-04 14:31:21 -07:00
A.J. Beamon
2568556360
Merge pull request #9895 from sfc-gh-ajbeamon/fix-chained-signal-handler-thread-race
...
Avoid a thread race trying to initialize the chained signal handler
2023-04-04 12:23:45 -07:00
Zhe Wang
8102ac9a41
Serialize concurrent datamoves and their cleanups ( #9421 )
...
* retry when concurrent dm cleanups happens
* cleanup
* fix conflict data move and cleanup
* clean
* make code looks better
* use background cleanup
* fmt
* unset StartMoveShardsFoundConflictingDataMove error
* address comments
2023-04-04 11:38:24 -07:00
A.J. Beamon
ebf3cc8743
Merge pull request #9897 from sfc-gh-ajbeamon/remove-unused-file
...
Remove MetaclusterManagement.actor.h
2023-04-04 11:38:02 -07:00
A.J. Beamon
503ecea17e
The MetaclusterManagement.actor.h file was meant to be removed as part of the metacluster refactor
2023-04-04 10:15:37 -07:00
Zhe Wu
90f035d7f7
Merge pull request #9894 from halfprice/zhewu/disable-gc-generation
...
Disable GC Tlog generation due to a bug exposed by the new logic
2023-04-04 10:05:37 -07:00
A.J. Beamon
9c786e6d1e
Merge pull request #9854 from sfc-gh-ajbeamon/metacluster-separate-project
...
Metacluster refactoring
2023-04-04 09:43:41 -07:00
A.J. Beamon
f08825fdea
Initialize the chained signal handler just in case it is possible that the new signal handler is installed prior to the old one being populated.
2023-04-04 09:26:07 -07:00
Zhe Wu
8d6ddecd82
Disable GC generation test
2023-04-04 08:56:39 -07:00
Jingyu Zhou
f17e466aa4
Remove printable() from TSS trace events ( #9889 )
2023-04-04 08:46:39 -07:00
Michael Stack
2cb76d06d1
The sharded rocks implementation skipped on TSS handling. Backfill. ( #9759 )
2023-04-04 08:35:30 -07:00
Zhe Wu
3751641704
Disable GC Tlog generation due to a bug exposed by the new logic in the existing code
2023-04-04 08:12:03 -07:00
Nim Wijetunga
2867443f95
Recurit EKP without Enabling Encryption ( #9885 )
...
Recruit EKP without needing to enable encryption
2023-04-03 20:05:21 -07:00
Hui Liu
f2a406f609
Add blob manifest and mutation log status to "status json" ( #9856 )
2023-04-03 18:30:13 -07:00
neethuhaneesha
1d6908d3b4
Changing single key deletions to delete based on number of deletes instead of bytelimit. ( #9867 )
2023-04-03 13:55:58 -07:00
Jingyu Zhou
047b1b1326
Merge pull request #9814 from sbodagala/main
...
FdbServer not able to join cluster
2023-04-03 13:15:55 -07:00
Ata E Husain Bohra
769226e5c0
EaR: Fix heap-over-flow in BlobCipherTest ( #9877 )
...
Description
Heap overflow was due to recent upgrade in BlobCipherTest to use
variable size 'baseCipher' buffer.
Testing
BlobCipherUnit test
2023-04-03 12:25:59 -07:00
Yanqin Jin
0faa6538d1
Remove an assertion for UB ( #9865 )
...
Original code comment says
"there's no hard guarantee about the above equality and the result could vary by platform, malloc implementation, and tooling instrumentation (e.g. ASAN, valgrind)"
Therefore, we should remove the assertion on UB. Since removal of the UB makes the rest of the test case
less meaningful, we commented out the entire test case.
Furthermore, we have a case in which, even without ASAN or other instrumentation, e.g. Valgrind, etc, the equality does not hold.
Profile: team
Commit hash: 3164cadc6f7bf6ed00e8d22ce8dcd51820a152a7
Command: devRetryCorrectnessTest bin/fdbserver -r simulation -f tests/fast/RandomUnitTests.toml -s 3939330597 -b on --crash --trace_format json
2023-04-03 12:01:25 -07:00
A.J. Beamon
c8f9e67822
Add a comment explaining Metacluster.h
2023-04-03 10:31:17 -07:00
Zhe Wu
7ab0ae8189
Merge pull request #9041 from halfprice/zhewu/gc-earlier-generations
...
Track TLog generation recovery and remove no longer needed TLog generations before recovery reaches to `fully_recovered`
2023-04-03 10:16:19 -07:00
neethuhaneesha
44fba4e84c
RocksDB 7.10.2 version upgrade ( #9828 )
2023-04-03 10:01:22 -07:00
A.J. Beamon
d9502b20bf
Avoid a thread race trying to initialize the chained signal handler and calling it when handling SIGPROF
2023-04-03 09:55:09 -07:00
Zhe Wu
c55cc23b47
Merge pull request #9873 from halfprice/zhewu/exclude-locality-check
...
Check if the locality is already excluded before issuing exclude locality transaction
2023-04-02 09:21:13 -07:00
Zhe Wu
c359f1686d
Update test document
2023-04-01 20:33:49 -07:00
Zhe Wu
14a0dbe4f9
Update CMakeLists to include all tests
2023-04-01 19:24:37 -07:00
Zhe Wu
414faa7fe2
Adding test to test exclude locality command
2023-04-01 19:04:58 -07:00
Zhe Wu
50a20946d1
Implement check if locality is already excluded in exclude locality command
2023-04-01 19:04:58 -07:00
Zhe Wu
bfd0c3a8e5
Merge pull request #9870 from halfprice/zhewu/update-test-config
2023-03-31 23:09:13 -07:00
Zhe Wu
074db3c646
Update SpecialKeySpaceRobustness config
2023-03-31 21:59:12 -07:00
Jingyu Zhou
b18da9e8da
Merge pull request #9868 from halfprice/zhewu/fix-exclude-bug
...
checkSafeExclusion should always create new ExclusionSafetyCheckRequest
2023-03-31 17:49:00 -07:00
Zhe Wu
d903d0cc8d
checkSafeExclusion should always create new ExclusionSafetyCheckRequest
2023-03-31 16:27:26 -07:00
Junhyun Shim
3c5dd71191
Allow token "aud" field to have a string as value ( #9860 )
2023-03-31 23:51:41 +02:00
Jingyu Zhou
92c182a842
Merge pull request #9858 from sfc-gh-nwijetunga/nim/fix-rest-connector-tls
...
Enable TLS with HTTPS connection
2023-03-31 10:02:58 -07:00
Jingyu Zhou
3560117f71
Merge pull request #9846 from johscheuer/fix-exclusion-check
...
Don't stop iterating over all storage processes in exclusion check
2023-03-31 09:59:00 -07:00
Ata E Husain Bohra
3f6fcada45
EaR - Misc fixes found using end-to-end integration testing ( #9806 )
...
* EaR - Misc fixes found using end-to-end integration testing
Description
Major changes proposed includes:
1. RESTClient filtering of trailing `/`(s) characters from
input URI resource path
2. Avoid EKP exponential backup given RESTClient supports
exponential backoffs retries for all retryable errors.
3. Memory allocation optimizations:
3.1. BaseCipher key management using Standalone semantics
in KMSConnector interface endpoints
3.2. Optimize memcpy while looking encryption-keys in EKP endpoints
4. Avoid delay while starting EKP, given its criticality during
cluster recovery.
5. Update BlobCipher to handle variable size BaseCipher buffer
6. Improved logging
Testing
Setup:
1. External KMS server to supply encryption keys (inhouse)
2. Create cluster with: cluster_aware & domain_aware config
* Fix EncryptionOps test
Description
Testing
* EaR - Misc fixes found using end-to-end integration testing
Description
Major changes:
1. Cleanup EKP driven exponential backup files.
2. Update EKP not to use #1 .
Testing
* EaR - Misc fixes found using end-to-end integration testing
Description
Address review comments
Testing
* Fix AES 256 key length value
Description
Testing
* Address review comments
Description
Testing
2023-03-30 22:22:26 -07:00
Nim Wijetunga
4a68e6072a
enable tls with https connection
2023-03-30 21:45:03 -07:00
A.J. Beamon
807646675c
Refactor the metacluster project into smaller files, and reorganize the namespaces. Move some metacluster and tenant testing helpers into the metacluster project.
2023-03-30 16:20:09 -07:00
