1
0
mirror of https://github.com/apple/foundationdb.git synced 2025-06-01 10:45:56 +08:00

18242 Commits

Author SHA1 Message Date
Andrew Noyes
96cbfe668c
Fix flaky ctest tests ()
* Use localhost cluster for trace_partial_file_suffix_test

This way we get a predictable 127.0.0.1 in the trace file name

* Skip suspend test of pidof is not available

* Avoid writing to closed trace log

calling fdb_network_stop sends a "close" message to the trace thread,
but the network thread might can still be running and sending "flush"
messages to the network thread. This change basically ignores any
flushes that come after a close.

* Ensure unique ports for multi-process tests
2022-01-28 13:16:44 -08:00
A.J. Beamon
213ddc9ac3
Merge pull request from sfc-gh-ajbeamon/fix-init-order
Fix constructor initialization order warning
2022-01-28 12:46:55 -08:00
Renxuan Wang
f9f3735f73 Add resolveHostnamesBlocking() in ConnectionString and IClusterConnectionRecord.
Also, combine IClusterConnectionRecord::getConnectionString() and IClusterConnectionRecord::getMutableConnectionString() to IClusterConnectionRecord::getConnectionString(), and rename setConnectionString() to setAndPersistConnectionString().
2022-01-28 12:20:41 -08:00
Renxuan Wang
2ea4146e1f Add resolveTCPEndpointBlocking() to resolve hostnames where async resolving is impossible. 2022-01-28 12:20:41 -08:00
gaozengqi
23ea6e2c7e Fix typo 2022-01-28 10:40:33 -08:00
gaozengqi
f1bd0f16da Run clang-format 2022-01-28 10:40:33 -08:00
gaozengqi
a0c0342e76 Fix usage 2022-01-28 10:40:33 -08:00
gaozengqi
0e6dbf434b Fix use allKeys.end 2022-01-28 10:40:33 -08:00
gaozengqi
1419d1aab7 Fix short hand 2022-01-28 10:40:33 -08:00
gaozengqi
697d075e20 Add role 'kvfiledump' to dump key-values from storage file 2022-01-28 10:40:33 -08:00
A.J. Beamon
0803abedaa Adjust the name order of constructor variable initialization to match declaration order. 2022-01-28 10:27:13 -08:00
Renxuan Wang
4a8e2a80e6 Improve/fix disk metrics.
1. Introduce processDiskReadSeconds and processDiskWriteSeconds, which stands for disk read/write times `since the last logging`. They can only be obtained on Linux and macOS, and will be 0 on Windows and FreeBSD;
2. Rename `busyTicks` to `IOMilliSecs`;
3. On FreeBSD, the metrics should be collected among all devices.
2022-01-27 14:40:32 -08:00
Lukas Joswiak
13da5d394b Remove extra StringRef copy 2022-01-26 19:22:40 -08:00
Lukas Joswiak
bd0dcfedb9 Re-enable CycleTestRestart downgrade tests 2022-01-26 19:22:40 -08:00
Chaoguang Lin
3dad130e72 Disable setclass test for now 2022-01-26 16:42:13 -08:00
Evan Tschannen
25c743d5a6
Merge pull request from sfc-gh-ajbeamon/change-feed-list-format
Use printable when formatting change feed ranges in fdbcli
2022-01-26 15:54:25 -08:00
A.J. Beamon
7989b83450 Use printable when formatting change feed ranges 2022-01-26 14:41:47 -08:00
Evan Tschannen
bb082344e2
Merge pull request from sfc-gh-ahusain/ahusain-encryptServerRole
Add new FDB EncryptKeyProxy role
2022-01-26 14:13:56 -08:00
Andrew Noyes
c967e6246b
Fix uninitialized memory in storage server () 2022-01-26 13:24:35 -08:00
A.J. Beamon
1800259d7c
Merge pull request from sfc-gh-ajbeamon/client-description-trace-field
Add ClientDescription field for client processes to identify the client logging an event.
2022-01-26 11:37:31 -08:00
Ata E Husain Bohra
f3c3ab06f1 Add new FDB EncryptKeyProxy role
diff-1: Address review comments

Major changes includes:

1. Add a new FDB role responsible- EncyrptKeyProxy. The role is
   responsible to expose APIs to fetch encyrption keys interacting
   with external Encryption KeyManager interface.
2. The process is a FDB singleton process following similar recruitment
   rules as other singleton processes in the system.
3. Code to recruit the worker process; given the encryption keys are
   needed during recovery (decode TLog records), for now the process
   is co-located in same datacenter as ClusterController.
4. Skeleton process actor code; more functionality will be added in
   subsequent PRs.

NOTE: The code is protected under a SERVER_KNOB with the default
      value as 'false' for now.:%s
2022-01-25 23:12:49 -08:00
Lukas Joswiak
e2e853a30b Use ObjectWriter to write storage cache data
This allows deserialization of the on disk data in lower FDB versions
that don't contain fields added in main.
2022-01-25 19:24:14 -08:00
Ata E Husain Bohra
87ee4cf958 Add new FDB EncryptKeyProxy role
Major changes includes:

1. Add a new FDB role responsible- EncyrptKeyProxy. The role is
   responsible to expose APIs to fetch encyrption keys interacting
   with external Encryption KeyManager interface.
2. The process is a FDB singleton process following similar recruitment
   rules as other singleton processes in the system.
3. Code to recruit the worker process; given the encryption keys are
   needed during recovery (decode TLog records), for now the process
   is co-located in same datacenter as ClusterController.
4. Skeleton process actor code; more functionality will be added in
   subsequent PRs.

NOTE: The code is protected under a SERVER_KNOB with the default
      value as 'false' for now.
2022-01-25 17:38:27 -08:00
Chaoguang Lin
9fba84bc45 Change SnapIncrementalRestore.toml to .txt 2022-01-25 14:41:35 -08:00
Chaoguang Lin
262e94637a Add the comment for the temporary change 2022-01-25 14:41:35 -08:00
Chaoguang Lin
7452f47f82 Disable downgrade test to 7.0 2022-01-25 14:41:35 -08:00
Chaoguang Lin
863ea8000c Split ConfigureTestRestart test to 7.0 and 7.1; remove [configuration] in SnapIncrementalRestore-1.toml; revert CycleTestRestart-1.toml to .txt 2022-01-25 14:41:35 -08:00
Chaoguang Lin
d0c21948a6 Revert "Use toml for snapshot test files"
This reverts commit 6b81065b32db0d8935fbfa862fc323eb7f6a0acd.
2022-01-25 14:41:35 -08:00
Yao Xiao
c605226a56
Add error handling in RocksDB KVS. () 2022-01-25 13:59:25 -08:00
leonliao
bc2c3df61f Fix wrong json path to check DB lock status while switching over dr clusters. 2022-01-24 13:37:09 -08:00
sfc-gh-tclinkenbeard
ef711d405c Add fdbbackup tags command 2022-01-24 13:34:54 -08:00
Ray Jenkins
ff0d1e5efa add back special_keys_api_failure error check 2022-01-24 13:32:44 -08:00
Ray Jenkins
d3055cc59a Use single transaction for setProcessClass and add fdbcli unit test. 2022-01-24 13:32:44 -08:00
Ray Jenkins
d1fe48ce84 Format fix. 2022-01-24 13:32:44 -08:00
Ray Jenkins
9f577a8675 Add state prefix to ThreadFuture. 2022-01-24 13:32:44 -08:00
Ray Jenkins
234760f516 Verify process network address filter before attempting setclass. Provide error message if no matching process addresses found 2022-01-24 13:32:44 -08:00
He Liu
5e0b1fd13f Resolved comments. 2022-01-21 16:58:42 -08:00
He Liu
0421562ce9 Resolved comments. 2022-01-21 16:58:42 -08:00
He Liu
b7388a714b Added get, scan, and commit counters. 2022-01-21 16:58:42 -08:00
He Liu
da8453eb5d Added systemClearRange. 2022-01-21 16:58:42 -08:00
He Liu
f14c832031 Added storage engine metrics in SS. 2022-01-21 16:58:42 -08:00
Andrew Noyes
0496f43dec
Merge pull request from sfc-gh-anoyes/anoyes/pc-guard-instrumentation
Add TRACE_PC_GUARD_INSTRUMENTATION_LIB
2022-01-21 15:00:50 -08:00
Renxuan Wang
334ba66348 Add the support for using hostname in simulation.
This PR does NOT turn the feature on. Also, disable hostname in downgrade restarting tests.
2022-01-21 10:48:33 -08:00
Josh Slocum
cf45354833 switched buggified and expected shard size for simulation 2022-01-20 20:37:03 -08:00
Josh Slocum
4bfef29e4c Changed small shards in simulation logic 2022-01-20 20:37:03 -08:00
Josh Slocum
6a8e9d71d2 Raising default minimum shard size, as it causes unecessary merging on growing clusters. 2022-01-20 20:37:03 -08:00
Steve Atherton
2384c5aeb9 Change Redwood default page size knob to 8192. 2022-01-20 20:31:26 -08:00
Renxuan Wang
8c49391c51 Address comments. 2022-01-20 19:18:15 -08:00
Renxuan Wang
02f72dd377 Address comments. 2022-01-20 19:18:15 -08:00
Renxuan Wang
1d62bc9437 Hostname needs a default constructor. 2022-01-20 19:18:15 -08:00