Andrew Noyes
1f541f02be
Merge branch 'anoyes/merge-6.2-to-6.3' into anoyes/release-6.3-merge
...
Merge, leaving conflict markers for now
2020-11-24 16:55:34 +00:00
Russell Sears
fc8b57189e
Merge pull request #3969 from Daniel-B-Smith/rocksdb-unsafe-fsync
...
RocksDB grab bag
2020-11-18 10:09:24 -08:00
David Youngworth
d0391db862
Merge branch 'release-6.2' into release-6.3
2020-11-16 10:15:23 -08:00
Xiaoge Su
3a6948c199
Report histogram periodically
2020-11-12 17:04:33 -08:00
Xiaoge Su
acbd4c1f39
Extract FETCH_KEYS_TOO_LONG_TIME_CRITERIA to Knob
2020-11-12 13:01:41 -08:00
Xiaoge Su
da31eb3d89
Periodically report the fetchKeys status
2020-11-09 15:24:05 -08:00
Daniel Smith
5825af272f
Fix whitespace
2020-11-04 22:06:30 +00:00
Steve Atherton
99c1880a83
Merge commit 'f9581de2005e6b085776e81b9fcaa16442b32589' into merge-6.2-to-6.3
...
# Conflicts:
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
2020-10-27 12:21:26 -07:00
Xin Dong
0bc51bb780
Resolve review comments
2020-10-26 16:44:50 -07:00
Xin Dong
c037bfd001
Added detailed logging when there is no servers left in a server team, because that may indicate a data loss incident.
2020-10-26 16:44:07 -07:00
Daniel Smith
44dc1303bb
Merge remote-tracking branch 'upstream/release-6.3' into rocksdb-unsafe-fsync
2020-10-26 19:24:33 +00:00
Meng Xu
b0d78ecf37
Merge pull request #3912 from dongxinEric/misc/log-detailed-teams-info-in-dd
...
Added a way to print detailed information about team collection for d…
2020-10-24 19:06:10 -07:00
Xin Dong
f757cae786
Address review comments
2020-10-23 14:01:53 -07:00
Xin Dong
8d0aa02a63
Do not periodically print detailed DD teams info
2020-10-16 16:11:14 -07:00
Jingyu Zhou
8f17a1a5d6
Merge branch 'release-6.2' into release-6.3
2020-10-16 15:25:39 -07:00
Xin Dong
92e31dd338
Address review comments
2020-10-15 15:25:00 -07:00
Xin Dong
1d43729cc9
Added a way to print detailed information about team collection for debugging.
2020-10-15 10:01:56 -07:00
Daniel Smith
a99b68a7e3
Add knob for tuning number of RocksDB read threads
2020-10-14 22:07:46 +00:00
Evan Tschannen
1378ecba4d
If a proxy is sufficiently far behind, reset network connections to attempt to fix the problem
2020-10-11 23:06:26 -07:00
Daniel Smith
6e287eb0d1
Merge remote-tracking branch 'upstream/release-6.3' into rocksdb-unsafe-fsync
2020-10-09 16:53:05 +00:00
Evan Tschannen
f546034366
do not prevent computePerOperation from being updated for small computeDurations. Added logging for the compute per operation. Protect against erroneously large compute estimates
2020-10-04 19:19:05 -07:00
Evan Tschannen
da26b0411c
increased the proxy commit memory limit
2020-10-04 19:16:51 -07:00
Meng Xu
6da7e56e03
FastRestore:Add more trace events
2020-09-22 19:57:19 -07:00
Meng Xu
17ece3d477
FastRestore:Fix FastRestoreApplierTransactionRateControl events
2020-09-22 13:25:43 -07:00
Meng Xu
002b1bec4c
FastRestore:Control write traffic at each applier
...
Controller assigns each applier a write rate.
Applier keeps the write-rate worth of transactions outstanding to DB.
This is to avoid heavily overloading DB while still keep enough
traffic to DB to get a good write throughput.
2020-09-22 08:14:28 -07:00
Evan Tschannen
fa306e07fd
temporarily disable hot read detection on the storage servers because it is causing a 3% to 5% performance regression in read heavy workloads
2020-08-31 09:30:51 -07:00
Evan Tschannen
29eec30183
Merge branch 'release-6.2' into release-6.3
...
# Conflicts:
# CMakeLists.txt
# build/Dockerfile
# build/Dockerfile.devel
# documentation/sphinx/source/downloads.rst
# fdbserver/Knobs.cpp
# fdbserver/LogSystem.h
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/TagPartitionedLogSystem.actor.cpp
# fdbserver/WaitFailure.actor.cpp
# fdbserver/fdbserver.vcxproj
# fdbserver/fdbserver.vcxproj.filters
# packaging/msi/FDBInstaller.wxs
2020-08-31 01:10:29 -07:00
Meng Xu
7a29a3157f
FastRestore:Fix various bugs discovered by enhanced simulation
...
1. sendMutation request can be dispatched when the version batch has finished and its data has been deleted;
2. Request scheduler on loader may get into infinite loop because FASTRESTORE_SCHED_LOAD_REQ_BATCHSIZE knob can be set to 0 in simulation
2020-08-26 15:53:18 -07:00
Meng Xu
bc766ab7ef
FastRestoreKnob:Adjust knobs
2020-08-26 09:54:33 -07:00
Meng Xu
996ba2374c
FastREstore:Fix:Incorrect condition in printing out FastRestoreLoaderSendMutationToApplierDoneTooLate
2020-08-24 17:27:40 -07:00
Meng Xu
6e3e36c8fc
FastRestore:RequestScheduler:Minor code style improvement
2020-08-24 10:45:46 -07:00
Meng Xu
a32cf7d293
FastRestore:Loader:Add sanity check before remove batch data
2020-08-18 15:44:22 -07:00
Evan Tschannen
8ede143941
Track tlog push latencies and reset connections if they are above 500ms
2020-08-18 08:43:14 -07:00
Meng Xu
7b7490efe7
FastRestore:Debug trace for seg fault
2020-08-17 20:34:33 -07:00
Meng Xu
01255b7ead
FastRestoreLoader:Sched:Add trace field and misc code style improvement
2020-08-16 09:43:04 -07:00
Meng Xu
cb0e44ff42
FastRestoreLoader:Sched send mutations per loadingParam requests
2020-08-16 07:38:51 -07:00
Meng Xu
98167a6d20
FastRestoreLoader:Fix:sched may stuck due to race condition
2020-08-16 06:29:09 -07:00
Meng Xu
015397be74
FastRestore:LoaderScheduler:Control number of load request to release at once
2020-08-15 22:18:48 -07:00
Meng Xu
a035e7a872
FastRestore:LoaderScheduler:Add knobs
2020-08-15 21:44:51 -07:00
Daniel Smith
94571786bf
Add RocksDB work
2020-08-14 17:16:20 +00:00
Xin Dong
b17667323e
Resolve review comments
2020-08-12 15:15:25 -07:00
Xin Dong
3ac6996844
Increased some knobs to throttle the spammy read hot logging. Also added more details inside the read hot log to make it useful
2020-08-12 14:27:34 -07:00
Daniel Smith
d625ab6368
Merge branch 'debug-logging-fix' into rocksdb-unsafe-fsync
2020-08-12 19:13:00 +00:00
Daniel Smith
9065fb3127
Add knob for periodic compactions
2020-08-11 18:00:01 +00:00
Meng Xu
87bde08695
FR:fix:sending samples may have broken promise error
...
The error should not skip the ack reply from loader to controller, saying
the file has been loaded.
Otherwise, restore controller can get stuck at waiting for the ack reply.
2020-08-06 10:41:18 -07:00
Meng Xu
7992cef025
FR:Fix sample network pkg can be too big
2020-08-04 22:35:21 -07:00
Meng Xu
2f5293fcc7
Introduce knob FASTRESTORE_USE_LOG_FILE and FASTRESTORE_USE_RANGE_FILE
2020-07-31 10:40:29 -07:00
Meng Xu
8cace30bb2
FastRestore:change TXN_BATCH_MAX_BYTES default to 1KB from 1MB
2020-07-30 16:43:35 -07:00
Daniel Smith
a94c4cce85
Add an unsafe option to disable manual fsyncing rocksdb
2020-07-30 22:31:18 +00:00
Meng Xu
ad915e462e
Add knob FASTRESTORE_NOT_WRITE_DB to skip writting to DB
2020-07-30 10:17:17 -07:00
