20780 Commits

Author SHA1 Message Date
Junhyun Shim
02b2f97e99
mTLS test helpers (#7218)
* Add TLS option to (local_cluster|tmp_cluster).py

* Add TLS-enabled C API test
2022-05-23 12:47:51 +02:00
Xiaoxi Wang
3fee0f33b5 change reference pointer 2022-05-23 00:12:48 -07:00
Xiaoxi Wang
27a60df36d fix rename bug 2022-05-22 23:37:32 -07:00
Xiaoxi Wang
0579b577ba remove polling interval; uncomment suppressFor 2022-05-22 23:35:39 -07:00
Xiaoxi Wang
909a7a92a1 rename preferLowerDiskUtil 2022-05-22 23:32:13 -07:00
Steve Atherton
c073f113a5
Explicitly cancel byte sample recovery on failure of a recovered storage server to prevent it from trying to read from the IKeyValueStore after shutdown. (#7212) 2022-05-20 19:55:52 -07:00
Kao Makino
a9d62ea031
Merge pull request #7169 from sfc-gh-khoxha/main
Use correct transaction latency in async mako
2022-05-20 15:49:39 -07:00
Dan Lambright
5e5e454362
Merge pull request #7199 from apple/blocking-peek-is-1sec-bug 2022-05-20 17:34:56 -04:00
Dan Lambright
801f6d52ee put buggigy knob conditionals in standard format 2022-05-20 16:02:07 -04:00
Xiaoxi Wang
5a431980d2 Merge branch 'main' of https://github.com/apple/foundationdb into features/debug-macro 2022-05-20 12:18:20 -07:00
Xiaoxi Wang
6b996dca29 add debug macro 2022-05-20 12:17:45 -07:00
Chaoguang Lin
5e8c3130e6 Extend the reboot interface to accept comma-delimited string;
Only fetch requested worker interfaces in rebootWorkerActor
Add killall test to multiprocess_fdbcli_tests
Make the rebootWorker only sends requests if all the requested workers are valid
Add comments; Organize print texts; Fix the bug clear the map every time run kill/suspend/expensive_data_check
2022-05-20 12:06:33 -07:00
Jon Fu
8ae951079f remove extra space for CI formatting 2022-05-20 13:28:26 -04:00
Jon Fu
612b94efa3 remove tenant_main arg and use CharsRef in error message format 2022-05-20 13:16:49 -04:00
Jon Fu
b92b3b21b0 Refactor Tenant class in fdb_api header to match conventions of existing classes and move tenant creation into populate function 2022-05-20 13:03:59 -04:00
Hao Fu
fc8d04b19d
Make java binding work for prefetch match index modes. (#7186)
* Make java binding work for prefetch match index modes.

It adds boundaryAndExist in return value, so that caller can see
whether a boundary index is orphan or not.

* Make Java code work with index match modes

* Add test for boundaryAndExist in java binding test
2022-05-20 09:10:02 -07:00
Ray Jenkins
dcd771bd07
Use semicolon for consistency with all other spans. (#7201) 2022-05-20 10:18:00 -05:00
Ray Jenkins
d1efc05a9b
Remove parentContext.traceID from MessagePack serialization. (#7197)
The OTEL receiver does not utilize the parent Trace id, only parent Span id. Essentially
it relies on Trace ids implicitly matching during trace assembly. Therefore, we may remove
these 16 bytes in serialization.
2022-05-19 16:43:06 -05:00
He Liu
78dc7c5d77 Resolving comments. 2022-05-19 13:27:21 -07:00
He Liu
54c566e57d Drop RocksDB column family before create a new column family from
import.
2022-05-19 13:27:21 -07:00
He Liu
61490022b4 Enabled checkpoint, restore test for RocksDB format. 2022-05-19 13:27:21 -07:00
He Liu
bc509d9572 Added fetchCheckpointKeyValuesQ in storage server. 2022-05-19 13:27:21 -07:00
He Liu
05bbe174c3 Implemented checkpoint(), restore() for KeyValueStoreRocksDB, for the
new format.
2022-05-19 13:27:21 -07:00
He Liu
5233d2a6ac ServerCheckpoint.actor.* 2022-05-19 13:27:21 -07:00
He Liu
f8dd9acc7f Cleaned up RocksDBCheckpointUtil* 2022-05-19 13:27:21 -07:00
He Liu
9797a2b663 Added StorageServerInterface::fetchCheckpointKeyValues API. 2022-05-19 13:27:21 -07:00
Xiaoxi Wang
4f28af1ab7 add comment 2022-05-19 10:50:24 -07:00
Trevor Clinkenbeard
d7f0928a48
Merge pull request #7161 from sfc-gh-bvr/ddspruce
Minor readability improvement (I believe) to DD code
2022-05-19 08:45:22 -07:00
Dan Lambright
a4fa68453c In version vector, blocking peek timeout cannot be much greater than max_read_transaction_life_versions 2022-05-19 10:34:37 -04:00
A.J. Beamon
993a79b42c
Restart protocol monitoring logic when we cycle the coordinator, even if its the same coordinator. Add some defensive code in case we fetch the protocol version for an absent peer. (#6397) 2022-05-19 10:00:23 +02:00
Xiaoxi Wang
6c11fc74ba add debug traces 2022-05-18 15:20:23 -07:00
Jon Fu
cdca68e26a only use tenant management prefix and default tenants to 0 in mako 2022-05-18 17:26:05 -04:00
Jon Fu
5a1c7f5c08 attempt to create fdb_api.cpp to separate static declaration 2022-05-18 17:16:30 -04:00
Xiaoxi Wang
7541fb9a25 address some review comments 2022-05-18 13:39:07 -07:00
Xiaoxi Wang
5355a31bda adjust team comparison inflight penalty 2022-05-18 13:21:06 -07:00
Jon Fu
cc6620e9bb fix CI errors 2022-05-18 16:03:44 -04:00
Trevor Clinkenbeard
b55b67641c
Fix -Wlogical-op-parentheses warning (#7190) 2022-05-18 12:31:43 -07:00
Hao Fu
dcacb30688
Add missing secondary queries tests for index prefetch (#7185)
* Add missing secondary queries tests for index prefetch

This change adds test for missing secondary queries for index prefetch,
in this case, MATCHED_ONLY mode would NOT return KV and UNMATCHED_ONLY
mode would return KV.

* remove default value for params
2022-05-18 12:13:19 -07:00
Jon Fu
17c8b4b2a0 Initial mako changes introducing Tenant API and replacing transaction resets 2022-05-18 14:48:40 -04:00
Xiaoxi Wang
8adf38ba08 Merge branch 'main' of https://github.com/apple/foundationdb into readaware 2022-05-18 10:11:18 -07:00
Hao Fu
f00606fd71
Enable MATCHED and UNMATCHED mode for index prefetch (#7162)
* Enable MATCHED and UNMATCHED mode for index prefetch

MATCHED mode returns index entries whose secondary KVs are present,
UNMATCHED mode returns index entries whose secondary KVs are absent.

Note that the conflict read range of this txn is set in 2 steps:
* Set the conflict range for primary query according to request
* Set the conflict ranges for secondary queries according to responses.

As a result, conflicts of different match_index mode are taken care of.

* Fix c binding
2022-05-18 09:16:28 -07:00
Ata E Husain Bohra
728869466d
Update EncryptBaseCipher cache to be index using {baseCipherId, domainId} (#7183)
Description

Major changes proposed in the patch includes:
1. Update EncryptKeyProxy EncyrptBaseCipherKeyId cache to be indexed
   using {encryptDomainId, baseCipherId} instead of only 'baseCipherId'
2. Enhance RESTKmsConnector 'error' tag to encapsulte: errorMessage
   and errorCode information

Testing

1. Updated EncyrptKeyProxy test
2. Updated RESTKmsConnector unit test
2022-05-18 06:16:40 -07:00
Vaidas Gasiunas
5205b565ab
Upgrade Tests: Testing downgrade from 7.2 to 7.1 (#7179) 2022-05-18 11:59:29 +02:00
Xiaoxi Wang
48e7689cc4 restore knob value 2022-05-17 22:20:35 -07:00
Xiaoxi Wang
dc03bd0a29
Merge pull request #7182 from sfc-gh-xwang/main
remove duplicate recoveryCount++
2022-05-17 16:20:51 -07:00
Xiaoxi Wang
e109d14771 Merge branch 'main' into readaware 2022-05-17 14:49:31 -07:00
Xiaoxi Wang
8950822e36 add knobs; change knobs 2022-05-17 14:49:27 -07:00
Xiaoxi Wang
9b28b46a1d remove duplicate recoveryCount++ 2022-05-17 14:10:01 -07:00
Ray Jenkins
7594f5c0f9
Mako add distributed tracing client (#7180)
* Add support for specifying distributed client tracer in Mako.

Introduces a new command line option --distributed_tracer_client. Defaults to disabled, user may specify
disabled, log_file, network_lossy, or sim_end.

Testing locally and verified with distributed tracing receiver. Note: We use a simple integer here
to denote tracer type rather directly using the TracerType enum in flow/Tracing.h. Including Tracing.h pulls in
FDBTypes.h and results in conflicts with several classes, including Database.
2022-05-17 15:23:14 -05:00
Josh Slocum
a43c98519d
Switching char* to std::string for ProcessInfo to have it own memory (valgrind errors) (#7176) 2022-05-17 12:41:04 -07:00