1
0
mirror of https://github.com/apple/foundationdb.git synced 2025-05-28 02:48:09 +08:00

409 Commits

Author SHA1 Message Date
Renxuan Wang
c69a07a858
Check in the new Hostname logic. ()
* Revert .

20220407-031010-renxuan-c101052c21da8346           compressed=True data_size=31004844 duration=4310801 ended=100000 fail_fast=10 max_runs=100000 pass=100000 priority=100 remaining=0 runtime=1:04:15 sanity=False started=100047 stopped=20220407-041425 submitted=20220407-031010 timeout=5400 username=renxuan

* Revert .

20220407-051532-renxuan-470f0fe6aac1c217           compressed=True data_size=30982370 duration=3491067 ended=100002 fail_fast=10 max_runs=100000 pass=100002 priority=100 remaining=0 runtime=0:59:57 sanity=False started=100141 stopped=20220407-061529 submitted=20220407-051532 timeout=5400 username=renxuan

* Revert .

Remove resolving-related functionalities in connection string. Connection string will be used for storing purpose only, and non-mutable.

20220407-175119-renxuan-55d30ee1a4b42c2f           compressed=True data_size=30970443 duration=5437659 ended=100000 fail_fast=10 max_runs=100000 pass=100000 priority=100 remaining=0 runtime=0:59:31 sanity=False started=100154 stopped=20220407-185050 submitted=20220407-175119 timeout=5400 username=renxuan

* Add hostname to coordinator interfaces.

* Turn on the new hostname logic.

* Add the corresponding change in config txns.

The most notable change is before calling basicLoadBalance(), we need to call tryInitializeRequestStream() to initialize request streams first.

Passed correctness tests.

* Return error when hostnames cannot be resolved in coordinators command.

* Minor fixes.
2022-04-27 21:54:13 -07:00
Renxuan Wang
ec95fb7e74 Disable hostname in simulations.
Hostnames are causing simulation failures. Turn on again in .
2022-04-25 11:05:25 -07:00
Bharadwaj V.R
ed08cfbf52
Merge branch 'apple:main' into block-down 2022-04-22 06:19:38 -07:00
Zhe Wang
6c9ff6ee5e
Add sharded rocksdb type ()
* add-sharded-rocksdb-type

* address comments

Co-authored-by: Zhe Wang <zhewang@Zhes-MacBook-Pro.local>
2022-04-21 22:53:14 -04:00
Bharadwaj V.R
e4f22e5394
Merge branch 'apple:main' into block-down 2022-04-12 10:50:02 -07:00
Markus Pilman
bf956f5630 Merge remote-tracking branch 'origin/main' into features/private-request-streams 2022-04-07 13:29:27 -06:00
Markus Pilman
e2d7d4075d multiple bug fixes 2022-04-07 11:08:07 -06:00
Bharadwaj V.R
58a6442d1f Add tracing and fix the version write and update logic 2022-04-06 21:06:01 -07:00
Chaoguang Lin
7d365bd1bb
Remote ikvs debugging ()
* initial structure for remote IKVS server

* moved struct to .h file, added new files to CMakeList

* happy path implementation, connection error when testing

* saved minor local change

* changed tracing to debug

* fixed onClosed and getError being called before init is finished

* fix spawn process bug, now use absolute path

* added server knob to set ikvs process port number

* added server knob for remote/local kv store

* implement simulator remote process spawning

* fixed bug for simulator timeout

* commit all changes

* removed print lines in trace

* added FlowProcess implementation by Markus

* initial debug of FlowProcess, stuck at parent sending OpenKVStoreRequest to child

* temporary fix for process factory throwing segfault on create

* specify public address in command

* change remote kv store knob to false for jenkins build

* made port 0 open random unused port

* change remote store knob to true for benchmark

* set listening port to randomly opened port

* added print lines for jenkins run open kv store timeout debug

* removed most tracing and print lines

* removed tutorial changes

* update handleIOErrors error handling to handle remote-ikvs cases

* Push all debugging changes

* A version where worker bug exists

* A version where restarting tests fail

* Use both the name and the port to determine the child process

* Remove unnecessary update on local address

* Disable remote-kvs for DiskFailureCycle test

* A version where restarting stuck

* A version where most restarting tests green

* Reset connection with child process explicitly

* Remove change on unnecessary files

* Unify flags from _ to -

* fix merging unexpected changes

* fix trac.error to .errorUnsuppressed

* Add license header

* Remove unnecessary header in FlowProcess.actor.cpp

* Fix Windows build

* Fix Windows build, add missing ;

* Fix a stupid bug caused by code dropped by code merging

* Disable remote kvs by default

* Pass the conn_file path to the flow process, though not needed, but the buildNetwork is difficult to tune

* serialization change on readrange

* Update traces

* Refactor the RemoteIKVS interface

* Format files

* Update sim2 interface to not clog connections between parent and child processes in simulation

* Update comments; remove debugging symbols; Add error handling for remote_kvs_cancelled

* Add comments, format files

* Change method name from isBuggifyDisabled to isStableConnection; Decrease(0.1x) latency for stable connections

* Commit the IConnection interface change, forgot in previous commit

* Fix the issue that onClosed request is cancelled by ActorCollection

* Enable the remote kv store knob

* Remove FlowProcess.actor.cpp and move functions to RemoteIKeyValueStore.actor.cpp; Add remote kv store delay to avoid race; Bind the child process to die with parent process

* Fix the bug where one process starts storage server more than once

* Add a please_reboot_remote_kv_store error to restart the storage server worker if remote kvs died abnormally

* Remove unreachable code path and add comments

* Clang format the code

* Fix a simple wait error

* Clang format after merging the main branch

* Testing mixed mode in simulation if remote_kvs knob is enabled, setting the default to false

* Disable remote kvs for PhysicalShardMove which is for RocksDB

* Cleanup #include orders, remove debugging traces

* Revert the reorder in fdbserver.actor.cpp, which fails the gcc build

Co-authored-by: “Lincoln <“lincoln.xiao@snowflake.com”>
2022-03-31 17:08:59 -07:00
He Liu
dd15489605 rename ssd-rocksdb-experimental as ssd-rocksdb-v1. 2022-03-29 10:53:38 -07:00
sfc-gh-tclinkenbeard
77786f4fc6 Merge remote-tracking branch 'origin/main' into change-data-hall 2022-03-27 12:44:05 -07:00
Josh Slocum
f27475e2f4 Merge branch 'main' into blob_integration 2022-03-22 11:41:58 -05:00
sfc-gh-tclinkenbeard
a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
Markus Pilman
35f7843d84 Merge remote-tracking branch 'origin/main' into features/private-request-streams 2022-03-21 12:33:53 +01:00
Josh Slocum
bf9bf19797 Disabling tenants for BlobGranule tests for now 2022-03-18 10:30:54 -05:00
Josh Slocum
37e7c80f26 Merge branch 'main' into blob_integration 2022-03-17 18:45:42 -05:00
A.J. Beamon
ee70837970 Remove some debugging lines, change simulated default cluster frequency, and use a different mechanism to disable tenants for the change feed test. 2022-03-17 12:10:19 -07:00
A.J. Beamon
9e39ebd069 Use the new enum name OPTIONAL_TENANT 2022-03-17 12:10:19 -07:00
A.J. Beamon
d0dc756c6d Allow disabling tenant mode in simulation. Fix a few bugs. 2022-03-17 12:10:18 -07:00
A.J. Beamon
81e8c7c362 Various test fixes to work with tenants. 2022-03-17 12:10:18 -07:00
A.J. Beamon
4b521b38cb Update various workloads to specify that they are expecting raw access. A couple other raw access related fixes. Disable tenant tests in backup and DR tests for now. 2022-03-17 12:10:18 -07:00
A.J. Beamon
05495908b8 Implement some tenant tests 2022-03-17 12:10:18 -07:00
Markus Pilman
118b53b7cf Merge remote-tracking branch 'origin/main' into features/private-request-streams 2022-03-17 12:06:44 +01:00
Steve Atherton
59762fc784
Merge pull request from sfc-gh-satherton/redwood-bug-fixes-and-memory-leak
Redwood memory usage and small memory leak fixes
2022-03-15 16:52:28 -07:00
Markus Pilman
117ee637db Merge remote-tracking branch 'origin/main' into features/private-request-streams 2022-03-15 17:17:47 +01:00
Markus Pilman
bed799220a Addressed review comments, added test 2022-03-15 16:57:26 +01:00
Yao Xiao
09df9d83fd
Enable RocksDB with Valgrind. () 2022-03-14 21:11:44 -07:00
Josh Slocum
4d909abbd1 re-enabling all storage engine types 2022-03-09 09:21:45 -06:00
Josh Slocum
e71b3533f9 Merge branch 'main' into blob_integration 2022-03-09 08:59:56 -06:00
A.J. Beamon
5fa9d3e1b7 Add a tenant parameter to read and commit requests. Store a map of all tenants on commit proxy and storage servers. Add an option to require tenant mode. 2022-03-06 21:54:21 -08:00
Steve Atherton
f03c0b8c3c Added ISimulated::restarted for detecting a restarted simulation test. 2022-03-04 17:19:46 -08:00
sfc-gh-tclinkenbeard
59794ffcb2 Add randomlyRenameZoneId test parameter 2022-03-01 00:42:24 -08:00
sfc-gh-tclinkenbeard
db8def68db Use std::unique_ptr for ISimulator::extraDB 2022-02-28 13:12:31 -08:00
A.J. Beamon
250a88e682 Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement. 2022-02-24 12:25:52 -08:00
Renxuan Wang
481587a8c6 Turn on hostname logic. 2022-02-22 16:29:59 -08:00
Josh Slocum
38a75a8b89 Merge branch 'main' into blob_integration 2022-02-17 17:47:38 -06:00
sfc-gh-tclinkenbeard
5b8f4d9be1 Move SimpleIni.h to fdbclient 2022-02-07 13:31:04 -08:00
Josh Slocum
4d34ac9545 disabling redwood and memory storage engines for tests for now 2022-02-07 08:15:45 -06:00
Renxuan Wang
334ba66348 Add the support for using hostname in simulation.
This PR does NOT turn the feature on. Also, disable hostname in downgrade restarting tests.
2022-01-21 10:48:33 -08:00
Suraj Gupta
968a4f9f50 Don't rely on database config to be updated. 2021-12-10 14:00:34 -06:00
Suraj Gupta
fc3376fe8f Move client knob to database config for blob granules. 2021-12-10 14:00:34 -06:00
Josh Slocum
efb21ca6a1 Merge branch 'master' into blob_integration 2021-11-24 13:17:54 -06:00
sfc-gh-tclinkenbeard
766a05d33c Merge remote-tracking branch 'origin/master' into add-format-warning 2021-11-17 12:14:01 -08:00
Renxuan Wang
725d31e264 Fix parameters passes to simulatedMachine(). 2021-11-16 17:50:12 -08:00
Steve Atherton
c53f5aa110 Renamed redwood to redwood-1-experimental and file extension to .redwood-v1. 2021-11-16 02:15:22 -08:00
Josh Slocum
f140028086 Merge branch 'master' into blob_integration 2021-11-15 15:09:40 -06:00
sfc-gh-tclinkenbeard
62efeb6812 Merge remote-tracking branch 'origin/master' into add-format-warning 2021-11-12 11:50:36 -08:00
Josh Slocum
ba53045fcd Fixing BW count for multi-process machines and having BW monitor its own removal since it doesn't get private mutations 2021-11-11 12:33:08 -06:00
sfc-gh-tclinkenbeard
25257f6f87 Enable unused-function warning for clang 2021-11-01 14:18:31 -07:00
sfc-gh-tclinkenbeard
13bb7838aa Enable clang -Wformat warning 2021-10-30 21:07:38 -07:00