apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-05-15 02:18:39 +08:00
Code Issues Packages Projects Releases Wiki Activity
13,756 Commits 35 Branches 318 Tags
Commit Graph

107 Commits

Author SHA1 Message Date
sfc-gh-tclinkenbeard
5b2e88b187 Use structured bindings in for loops 2020-12-27 01:46:20 -04:00
sfc-gh-tclinkenbeard
19816ccdbf Improve DataDistribution const-correctness 2020-12-26 20:22:27 -04:00
Andrew Noyes
877997632d Merge branch 'release-6.3' into anoyes/merge-release-6.3-master 
Include conflict markers for review purposes
 2020-12-04 01:38:07 +00:00
Andrew Noyes
dc2bac5670 Resolve conflicts 2020-11-24 19:09:42 +00:00
Andrew Noyes
1f541f02be Merge branch 'anoyes/merge-6.2-to-6.3' into anoyes/release-6.3-merge 
Merge, leaving conflict markers for now
 2020-11-24 16:55:34 +00:00
Evan Tschannen
2e13aacf1e
Merge pull request #4049 from sfc-gh-tclinkenbeard/fix-shard-merge-too-soon 
Fix HasBeenTrueFor::set function
 2020-11-18 10:29:58 -08:00
David Youngworth
d64cf8b9e3 Merge branch 6.3 into master 2020-11-17 11:22:45 -08:00
sfc-gh-tclinkenbeard
84632d63ad Remove stale comment 2020-11-16 17:54:47 -08:00
sfc-gh-tclinkenbeard
898fbac191 Remove special handling of broken_promise errors 2020-11-16 13:28:08 -08:00
sfc-gh-tclinkenbeard
4f369e70ab Add code coverage macro to DataDistributionTracker::SafeAccessor::operator() 2020-11-16 13:25:39 -08:00
sfc-gh-tclinkenbeard
6235d087a6 Prevent shardTracker or trackShardBytes from accidentally unsafely accessing DataDistributionTracker 2020-11-16 12:46:21 -08:00
David Youngworth
d0391db862 Merge branch 'release-6.2' into release-6.3 2020-11-16 10:15:23 -08:00
sfc-gh-tclinkenbeard
ca8ea3b6ff Fix memory issues caused by cancelling data distribution tracker 2020-11-15 23:52:36 -08:00
sfc-gh-tclinkenbeard
6917da9ce7 Fix HasBeenTrueFor::set 2020-11-13 13:15:51 -08:00
Meng Xu
222da17558 Merge branch 'release-6.2' into mengxu/ha-code-read 2020-11-12 13:39:27 -08:00
Meng Xu
c2dd7d1d38 Remove unresolved questions 2020-11-11 22:39:11 -08:00
Andrew Noyes
f467524e06 Don't dereference self on broken_promise 2020-11-11 00:24:23 +00:00
sfc-gh-tclinkenbeard
4669f837fa Add uses of makeReference 2020-11-07 22:10:18 -08:00
Meng Xu
063700e4d6 Add comments and questions to HA and tLog code reading 
The comments' correctness need to be confirmed by reviewers.
 2020-10-30 12:14:57 -07:00
Jingyu Zhou
8f17a1a5d6 Merge branch 'release-6.2' into release-6.3 2020-10-16 15:25:39 -07:00
sfc-gh-tclinkenbeard
91a8367acb Avoid slow task in ~DataDistributionTracker 2020-10-01 11:44:55 -07:00
Xin Dong
3ac6996844 Increased some knobs to throttle the spammy read hot logging. Also added more details inside the read hot log to make it useful 2020-08-12 14:27:34 -07:00
Chaoguang Lin
5bc2e2e595 update comments, make isAsync() virtual, remove unused code 2020-06-12 10:12:44 -07:00
Chaoguang Lin
980bee1d13 Fix fetchShardMetricsList_impl and add read cache in special key space 2020-06-12 10:12:44 -07:00
Chaoguang Lin
298b94a044 clang-format the change 2020-05-19 20:51:02 -07:00
Chaoguang Lin
c230c23a3a Add test for smaller range read in dd-stats 2020-05-19 20:49:15 -07:00
Chaoguang Lin
93ca130602 Delete the unnecessay endKey field in DDMetricsRef, since each begin key is the previous end key 2020-05-19 13:32:42 -07:00
Chaoguang Lin
ef724bf939 Merge remote-tracking branch 'upstream/master' into add-data-distribution-metrics 2020-05-08 18:39:28 -07:00
chaoguang
e8b62e48f4 Rename DDMetrics to DDMetricsRef 2020-05-08 17:17:27 -07:00
Xin Dong
1927f15898 Now the ReadHotDetection tests can be run reliably. Added it to nightly bundle. 2020-04-28 13:26:02 -07:00
Xin Dong
76eb1c4376 Address review comments from Meng 2020-04-21 11:00:19 -07:00
Xin Dong
7dd7406c59
Merge branch 'master' into feature/hot-read-key-detection-part-2 2020-04-16 14:54:05 -07:00
A.J. Beamon
b1172417f5 Merge branch 'master' into per-priority-busy-logging 
# Conflicts:
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/Net2.actor.cpp
 2020-04-14 14:22:12 -07:00
tclinken
247ab84323 Merge branch 'master' of https://github.com/apple/foundationdb into add-data-distribution-metrics 2020-03-23 17:01:17 -07:00
Evan Tschannen
e08f0201f1 merge release 6.2 into master 2020-03-17 12:51:47 -07:00
Evan Tschannen
7adc916e18
Merge pull request #2806 from ajbeamon/improve-team-request-performance 
Improve performance of get team requests.
 2020-03-16 11:56:45 -07:00
Evan Tschannen
12f2b32770 added additional logging in data distribution 2020-03-13 15:19:33 -07:00
A.J. Beamon
555db50cd1 Avoid calling into SABTF so frequently. Use a cheaper call that only checks that shards exist. 2020-03-12 11:22:03 -07:00
A.J. Beamon
abb75f7eb7 Add logging to indicate the time spent at each priority that exceeds some minimum busyness threshold 2020-02-07 14:34:24 -08:00
tclinken
c9363e7e28 Merge branch 'master' of https://github.com/apple/foundationdb into add-data-distribution-metrics 2020-01-22 21:02:21 -08:00
Xin Dong
b0a1af1288 Added the actual read hot detection algorithm and logging machanism. 
- When a shard has a read bandwidth larger than a threshold value(configurable via knob), and it's read-bandwidth/byte-size ratio is also larger than a threshold(configurable via knob), the corresponding shard tracker will run the algorithm
- The algorithm will divide the shard into 10MB(configurable via knob) chunks and try to find the chunk(s) that has large aforementioned ratio
- Then those ranges will be logged into TraceEvents. This will later do more like actually cache them.
 2020-01-21 11:19:52 -08:00
Xin Dong
1d6cd1007b Instead of using absolute value as the max bytesReadPerKSec threshold, use a pre-defined read traffic to byte size ratio to decide that value dynamically based on the actual size of the shard. 2020-01-21 11:15:52 -08:00
Evan Tschannen
3f9d9d8b84 Merge branch 'release-6.2' 
# Conflicts:
#	CMakeLists.txt
#	cmake/FlowCommands.cmake
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/StorageServerInterface.h
#	fdbserver/DataDistributionTracker.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	flow/Knobs.h
#	flow/Platform.cpp
#	versions.target
 2020-01-16 18:37:47 -08:00
tclinken
1d6ac716a1 Merge remote-tracking branch 'origin' into add-data-distribution-metrics 2020-01-15 13:20:04 -08:00
Evan Tschannen
fd5705a451 fixed capitalization 2020-01-15 09:35:57 -08:00
Evan Tschannen
c93ca04ea6 Do not merge more than 100 shards together to avoid creating untrackable shards 2020-01-15 09:33:27 -08:00
Evan Tschannen
b331c5dafe wantsToMerge was created before the shardEvaluator has a chance to update it based on shardSize changes 2020-01-10 17:23:56 -08:00
Evan Tschannen
fde53cbeef HasBeenTrueFor was ready immediately after a previous shard merge 2020-01-10 16:28:56 -08:00
Evan Tschannen
9b80498180 Added a trace event to warn if a shard is merged before enough time has elapses from becoming low bandwidth 2020-01-10 14:58:38 -08:00
Evan Tschannen
e4fa4ad0c9 Data distribution will not merge a shard unless it has been low bandwidth for 5 minutes 2020-01-09 17:02:49 -08:00