He Liu
bc5bfaffda
Shard based move ( #6981 )
...
* Shard based move.
* Clean up.
* Clear results on retry in getInitialDataDistribution.
* Remove assertion on SHARD_ENCODE_LOCATION_METADATA for compatibility.
* Resolved comments.
Co-authored-by: He Liu <heliu@apple.com>
2022-07-07 20:49:16 -07:00
Xiaoxi Wang
ef0f415e3d
add option; change to shared_ptr
2022-06-13 13:55:48 -07:00
Xiaoxi Wang
1de6c09307
use struct instead of tuple
2022-06-13 11:27:50 -07:00
Xiaoxi Wang
c12a7a30ed
Update fdbserver/DataDistributionQueue.actor.cpp
...
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
2022-06-13 08:22:48 -07:00
Xiaoxi Wang
b99bd45730
format code
2022-06-09 12:36:20 -07:00
Xiaoxi Wang
e5aa5fef22
merge upstream/main
2022-06-09 12:17:27 -07:00
Xiaoxi Wang
6ab12ea971
add storeTuple and unit test; refactor getSourceServersForRange
2022-06-09 12:16:12 -07:00
Bharadwaj V.R
24b2be1c4f
Remove last-limited check from DDMountainChopper and DDValleyFiller
2022-05-30 21:57:34 -07:00
Xiaoxi Wang
74748c20a0
a.fix heap-use-after-free caused by early noErrorsActors destroy
2022-05-27 10:14:10 -07:00
Xiaoxi Wang
13a77dd5a2
change priority knob; change PromiseStream to FutureStream; remove comments; add on_sr check
2022-05-25 17:09:34 -07:00
Xiaoxi Wang
73624bcd2a
Merge remote-tracking branch 'upstream/main' into readaware
2022-05-23 11:17:38 -07:00
Xiaoxi Wang
54e56af3db
solve some review comments
2022-05-23 11:04:37 -07:00
Xiaoxi Wang
3fee0f33b5
change reference pointer
2022-05-23 00:12:48 -07:00
Xiaoxi Wang
27a60df36d
fix rename bug
2022-05-22 23:37:32 -07:00
Xiaoxi Wang
0579b577ba
remove polling interval; uncomment suppressFor
2022-05-22 23:35:39 -07:00
Xiaoxi Wang
4f28af1ab7
add comment
2022-05-19 10:50:24 -07:00
Xiaoxi Wang
8950822e36
add knobs; change knobs
2022-05-17 14:49:27 -07:00
Xiaoxi Wang
f40a2a68fc
add more statistics
2022-05-17 10:19:09 -07:00
Xiaoxi Wang
3b241955e7
add more informative trace info
2022-05-16 21:25:56 -07:00
Xiaoxi Wang
dc5d2bff06
make ratekeeper don't limit read rebalance
2022-05-16 11:38:38 -07:00
Bharadwaj V.R
30922268d7
Minor readability improvement (I believe) to DD code; mostly, replacing .first, .second of pairs with better names through bindings)
2022-05-13 13:03:00 -07:00
Xiaoxi Wang
e9e11bf53b
change all criteria to knobs
2022-05-12 16:30:21 -07:00
Xiaoxi Wang
73e655a067
adjust (srcLoad - destLoad) / 10.0
2022-05-09 16:11:38 -07:00
Xiaoxi Wang
b0a00effa0
set max shard bandwidth
2022-05-06 16:37:12 -07:00
Xiaoxi Wang
03607675e8
refactor GetTeamRequest
2022-05-04 17:42:49 -07:00
Xiaoxi Wang
964e0cecef
format
2022-05-04 14:52:39 -07:00
Xiaoxi Wang
22eafcf7a2
rename trace event
2022-05-04 14:27:42 -07:00
Xiaoxi Wang
ae66ed6c16
fix DataDistributionQueue time_out ; reset the rebalance poll time
2022-05-04 14:11:20 -07:00
Xiaoxi Wang
a3d0b005dc
reset several method use getShardMetrics
2022-05-04 00:00:03 -07:00
Xiaoxi Wang
7c37d172b9
solve some comments
2022-05-03 17:21:08 -07:00
Xiaoxi Wang
00b97ec829
add storage metric compare knob; timeThrottle with constant
2022-04-27 23:37:35 -07:00
Xiaoxi Wang
73c7240bfd
code format
2022-04-26 17:34:34 -07:00
Xiaoxi Wang
9b211c48ee
format code
2022-04-26 17:13:37 -07:00
Xiaoxi Wang
8cc6d9aaac
determine timeThrottle and topK dynamically on the shard number
2022-04-25 16:59:20 -07:00
Xiaoxi Wang
cc05f5e9db
fix getMetrics keys bug
2022-04-24 21:21:18 -07:00
Xiaoxi Wang
a0b45c2962
fix throttle bug
2022-04-24 15:51:18 -07:00
Xiaoxi Wang
eefd0778c5
change canQueue to timeThrottle()
2022-04-22 15:26:44 -07:00
Xiaoxi Wang
4818f01de9
fix top10 shard index bug; add event detail; fix merge conflict
2022-04-22 14:14:58 -07:00
Xiaoxi Wang
0639810b66
merge upstream/main
2022-04-22 11:09:15 -07:00
Xiaoxi Wang
eb949ee3dc
change default shard size; enable valley filler best; top 10 random choice
2022-04-22 10:07:12 -07:00
Xiaoxi Wang
04311d001e
topK shard random selection
2022-04-21 22:37:16 -07:00
Xiaoxi Wang
960a13f233
format
2022-04-21 21:40:49 -07:00
Xiaoxi Wang
f9e57396e2
completeDest
2022-04-21 21:23:42 -07:00
Xiaoxi Wang
a8bfd0f481
move canQueue, 60s each source server, random select portion of shards/destServer
2022-04-20 23:12:34 -07:00
Xiaoxi Wang
af9e5ba885
move canQueue, 60s each source server, random select portion of shards
2022-04-20 22:19:56 -07:00
Xiaoxi Wang
131adec811
add canQueue
2022-04-20 15:28:03 -07:00
Xiaoxi Wang
25e1e75d9e
enable destComplete
2022-04-20 13:32:04 -07:00
Xiaoxi Wang
e0ae0942c4
fix inflight read division; temp destComplete fix; 0.1 constant poll time
2022-04-20 12:15:40 -07:00
Xiaoxi Wang
a7a9de781e
update unittests
2022-04-16 22:51:55 -07:00
Xiaoxi Wang
7b9432bdda
revert to original BgDDValleyFiller and BgDDMountainChopper for disk rebalance
2022-04-12 18:01:28 -07:00