apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-06-03 03:41:53 +08:00
Code Issues Packages Projects Releases Wiki Activity
20,221 Commits 35 Branches 318 Tags
74ec6962b0
Commit Graph

9205 Commits

Author SHA1 Message Date
Lukas Joswiak
0ae0bf9e31 Add proxy debug ID 2022-04-28 10:48:57 -04:00
Lukas Joswiak
0dc662bfd2 Move trace below where is updated 2022-04-28 10:48:57 -04:00
Lukas Joswiak
8b489ae0bf Lower max compute duration cutoff and increase suppression duration 2022-04-28 10:48:57 -04:00
Lukas Joswiak
802c45f843 Add estimated compute duration 2022-04-28 10:48:57 -04:00
Lukas Joswiak
f13a8df5d9 Add logging measuring commit compute duration 2022-04-28 10:48:57 -04:00
Xiaoxi Wang
02fc80f12e
Merge pull request from sfc-gh-xwang/xwang/main/ppw-bug-fix 
Fix excludedServer overwritten bug and correct some trace event field
 2022-04-27 23:23:27 -07:00
Renxuan Wang
c69a07a858
Check in the new Hostname logic. () 
* Revert .

20220407-031010-renxuan-c101052c21da8346           compressed=True data_size=31004844 duration=4310801 ended=100000 fail_fast=10 max_runs=100000 pass=100000 priority=100 remaining=0 runtime=1:04:15 sanity=False started=100047 stopped=20220407-041425 submitted=20220407-031010 timeout=5400 username=renxuan

* Revert .

20220407-051532-renxuan-470f0fe6aac1c217           compressed=True data_size=30982370 duration=3491067 ended=100002 fail_fast=10 max_runs=100000 pass=100002 priority=100 remaining=0 runtime=0:59:57 sanity=False started=100141 stopped=20220407-061529 submitted=20220407-051532 timeout=5400 username=renxuan

* Revert .

Remove resolving-related functionalities in connection string. Connection string will be used for storing purpose only, and non-mutable.

20220407-175119-renxuan-55d30ee1a4b42c2f           compressed=True data_size=30970443 duration=5437659 ended=100000 fail_fast=10 max_runs=100000 pass=100000 priority=100 remaining=0 runtime=0:59:31 sanity=False started=100154 stopped=20220407-185050 submitted=20220407-175119 timeout=5400 username=renxuan

* Add hostname to coordinator interfaces.

* Turn on the new hostname logic.

* Add the corresponding change in config txns.

The most notable change is before calling basicLoadBalance(), we need to call tryInitializeRequestStream() to initialize request streams first.

Passed correctness tests.

* Return error when hostnames cannot be resolved in coordinators command.

* Minor fixes.
 2022-04-27 21:54:13 -07:00
Xiaoxi Wang
c8a8ebd150 Merge branch 'main' of https://github.com/apple/foundationdb into xwang/main/ppw-bug-fix 2022-04-27 20:46:07 -07:00
A.J. Beamon
79063d5fc4
Better accounting for tenant prefixes when working with key size limits () 
* Consider tenant prefix sizes when doing key size limit checks

* Fix formatting
 2022-04-27 14:38:59 -07:00
Xiaoxi Wang
1f046e4be5 fix excludedServer overwritten bug; change some trace event field 2022-04-27 13:05:02 -07:00
Ata E Husain Bohra
333aadb903
Interface to enable clients to send/receive REST requests/responses () 
* Interface to enable clients to send/receive REST requests/responses

Description

Major changes:
1. Add RESTClient interface enabling client to send/receive REST HTTP
   requests. Support REST APIs are: get, head, put, post, delete, trace
2. Add RESTUtil file introducing below interfaces:
 2.1. RESTUrl - Extract URI information: host, service, request-parameters.
 2.2. RESTConnectionPool-
      Connection establishment, life-cycle management, connection-pool (TTL)
 2.3. RESTClientKnobs - supports REST Knob parameter management and updates

Testing

Unit test - fdbrpc/RESTClient, fdbrpc/RESTUtils
 2022-04-27 12:17:52 -07:00
Jingyu Zhou
23b51e34fd Fix spurious test failure 
The timed_out error, if happens, could cause the test to fail, even though
there is nothing wrong with the database.
 2022-04-26 09:40:43 -07:00
Steve Atherton
fe338ced8f
Merge pull request from sfc-gh-mpilman/features/validate-trace-events-in-simulation 
Reduce excessive tracing and fail after 1M traces
 2022-04-25 18:02:59 -07:00
Markus Pilman
cbe4a873d2 Merge remote-tracking branch 'origin/main' into features/validate-trace-events-in-simulation 2022-04-25 17:39:29 -06:00
Renxuan Wang
ec95fb7e74 Disable hostname in simulations. 
Hostnames are causing simulation failures. Turn on again in .
 2022-04-25 11:05:25 -07:00
Ray Jenkins
1c5bf135d5
Revert "Migrate to OpenTelemetry tracing. ()" () 
This reverts commit 5df3bac110d9b5b88931b008b852433688bb7eb0.
 2022-04-25 09:29:56 -05:00
Evan Tschannen
a40c8f231a fix: perpetual wiggle would not react to a pause that happened while starting the wiggle 2022-04-24 20:35:02 -07:00
Bharadwaj V.R
588b2fa509
Merge branch 'main' into block-down 2022-04-22 14:53:09 -07:00
Bharadwaj V.R
988a70f064
Merge pull request from sfc-gh-bvr/dbcorever 
Track newest and lowest compatible protocol versions in DBCoreState
 2022-04-22 14:46:21 -07:00
Bharadwaj V.R
4a5c2268da
Merge branch 'apple:main' into block-down 2022-04-22 14:45:54 -07:00
Markus Pilman
9e65e15b45 Merge remote-tracking branch 'origin/main' into features/validate-trace-events-in-simulation 2022-04-22 15:39:55 -06:00
Ata E Husain Bohra
6c9030408e Fix Build: use boost::hash to compute hash for std::pair 
Description

Fix Build: use boost::hash to compute hash for std::pair

Testing

1. Build - gcc/clang
2. Simulation test: EncryptKeyProxyTest, EncryptionOps
3. Unit test: flow/BlobCipher
4. Running 10k correctness Joshua run
 2022-04-22 13:16:30 -07:00
Sreenath Bodagala
706164d8ae - Address seg fault in "storageRecruiter" actor 2022-04-22 12:38:59 -07:00
Xiaoxi Wang
b8fbd18199
Merge pull request from sfc-gh-xwang/main 
Run completeDest after clear destination set
 2022-04-22 09:23:36 -07:00
Ata E Husain Bohra
670d40ef79
FDB native KMS Connector Framework () 
* FDB native KMS Connector Framework

Description

Major changes includes:
1. Framework code to enable FDB native KMS connector implementation.
2. SERVER_KNOBS->KMS_CONNECTOR_TYPE controls the connector type selection.
3. KmsConnectorInterface endpoint definitions, every KMSConnector
   implementation needs to support defined endpoints.
4. Update EncryptKeyProxy to leverage KmsConnectorInterface endpoints
   to fetch encryption keys on-demand and/or periodic refreshes.
   Integrate SimKmsConnector implementation.
5. Implement SimKmsConnector by leveraging existing SimKeyProxy
   implementation.

Testing

Unit test: fdbserver/SimKmsConnector
Simulation: EncryptKeyProxy
 2022-04-22 08:53:39 -07:00
Bharadwaj V.R
822eb9ec26
Merge branch 'apple:main' into dbcorever 2022-04-22 08:08:34 -07:00
Bharadwaj V.R
ed08cfbf52
Merge branch 'apple:main' into block-down 2022-04-22 06:19:38 -07:00
A.J. Beamon
1352083d4c
Merge pull request from sfc-gh-clin/deprecate-speical-keys 
Remove the client profiling special keys and update related documentations
 2022-04-21 21:51:16 -07:00
Xiaoxi Wang
960a13f233 format 2022-04-21 21:40:49 -07:00
Xiaoxi Wang
f9e57396e2 completeDest 2022-04-21 21:23:42 -07:00
Zhe Wang
6c9ff6ee5e
Add sharded rocksdb type () 
* add-sharded-rocksdb-type

* address comments

Co-authored-by: Zhe Wang <zhewang@Zhes-MacBook-Pro.local>
 2022-04-21 22:53:14 -04:00
Ata E Husain Bohra
04ecd8e08f
Revert "Revert "Update 'salt' details for EncryptHeader AuthToken details ()" ()" () 
Description

Major changes proposed:
1. This reverts commit f38b2e82098f432bc4437f823c5c1dafd52e1319.
2. Also add fix for Valgrind failure due to unintialized variables.
3. Improve checks to catch is cipherKey details cached in BlobCipherKeyCache
   isn't as expected

Testing

Overall correctness: 10K (20220421-193911-ahusain-foundationdb-a730e5cb38541e20)
EncyrptionOps correctness: 100K (20220421-194315-ahusain-foundationdb-29c598a8b9420430)
EncryptionOps Valgrind: 100 (20220421-194434-ahusain-foundationdb-7fc5f98eddc0921a)
 2022-04-21 18:57:56 -07:00
Ata E Husain Bohra
5b5b45c7cb
Update EKPServer LookupKeysByKeyIds API () 
* Update EKPServer LookupKeysByKeyIds API

Description

Major changes include:
1. Update EKPServer LookupKeysByKeyIds API to include
   'encryptDomainId' as input along with 'baseCipherId'
2. Unify EKPServer response definition to return
   std::vector<EKPBaseCipherDetails>
3. Update SimKmsProxy interfaces as per above mentioned
   changes.

Testing

1. Update EncryptKeyProxyTest simulation test.
2. Update SimEncyrptKeyProxy unit test
 2022-04-21 14:56:09 -07:00
Tao Lin
98b82f1ac3 Fix GetMappedRange test when rangeResult has more 2022-04-21 10:16:27 -07:00
Bharadwaj V.R
449a315c06
Merge branch 'apple:main' into block-down 2022-04-21 09:37:42 -07:00
Bharadwaj V.R
c20fb6ef6d
Merge branch 'apple:main' into dbcorever 2022-04-21 09:37:29 -07:00
Markus Pilman
f38b2e8209
Revert "Update 'salt' details for EncryptHeader AuthToken details ()" () 
This reverts commit a38318a6acc1ef71ebcc5ea4af6bb2ea0edf0887.
 2022-04-21 09:04:40 -07:00
Markus Pilman
bbb1392aad Merge remote-tracking branch 'origin/main' into features/validate-trace-events-in-simulation 2022-04-21 08:24:18 -06:00
Renxuan Wang
e40cc8722c
A few hostname improvements. () 
* Add tryResolveHostnames() in connection string.

* Add missing hostname to related interfaces.

* Do not pass RequestStream into *GetReplyFromHostname() functions.

Because we are using new RequestStream for each request anyways. Also, the passed in pointer could be nullptr, which results in seg faults.

* Add dynamic hostname resolve and reconnect intervals.

* Address comments.
 2022-04-20 13:42:46 -07:00
Markus Pilman
3335b2686e
Merge branch 'main' into features/validate-trace-events-in-simulation 2022-04-20 12:03:33 -06:00
Binglin Chang
408c0cf1c9
Fix compile errors on ubuntu 20.04 () 2022-04-20 10:00:46 -07:00
Bharadwaj V.R
8a0ce5bfc7 Rename isInvalidMagic and fix formatting 2022-04-20 09:17:28 -07:00
Bharadwaj V.R
96ab7c0254
Update fdbserver/Status.actor.cpp 
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
 2022-04-20 08:52:14 -07:00
Bharadwaj V.R
4d6f4ecd9c
Merge branch 'main' into dbcorever 2022-04-20 08:23:34 -07:00
Ray Jenkins
5df3bac110
Migrate to OpenTelemetry tracing. () 2022-04-20 09:26:37 -05:00
Bharadwaj V.R
a711c55061
Merge branch 'apple:main' into dbcorever 2022-04-20 06:16:27 -07:00
Bharadwaj V.R
89af5561f1
Merge branch 'apple:main' into block-down 2022-04-20 06:13:01 -07:00
Markus Pilman
d4ee7be1d7 Reduce excessive tracing and fail after 1M traces 2022-04-19 21:11:51 -06:00
Evan Tschannen
442d2b34c7
fix: pops which were ignored during a snapshot would not be replayed on the proper tlogs within a shared tlog () 2022-04-19 16:57:41 -07:00
Bharadwaj V.R
51ef860612
Merge branch 'apple:main' into block-down 2022-04-19 10:16:56 -07:00