253 Commits

Author SHA1 Message Date
Xiaoxi Wang
f40a2a68fc add more statistics 2022-05-17 10:19:09 -07:00
Xiaoxi Wang
3b241955e7 add more informative trace info 2022-05-16 21:25:56 -07:00
Xiaoxi Wang
dc5d2bff06 make ratekeeper don't limit read rebalance 2022-05-16 11:38:38 -07:00
Xiaoxi Wang
e9e11bf53b change all criteria to knobs 2022-05-12 16:30:21 -07:00
Xiaoxi Wang
73e655a067 adjust (srcLoad - destLoad) / 10.0 2022-05-09 16:11:38 -07:00
Xiaoxi Wang
b0a00effa0 set max shard bandwidth 2022-05-06 16:37:12 -07:00
Xiaoxi Wang
03607675e8 refactor GetTeamRequest 2022-05-04 17:42:49 -07:00
Xiaoxi Wang
964e0cecef format 2022-05-04 14:52:39 -07:00
Xiaoxi Wang
22eafcf7a2 rename trace event 2022-05-04 14:27:42 -07:00
Xiaoxi Wang
ae66ed6c16 fix DataDistributionQueue time_out ; reset the rebalance poll time 2022-05-04 14:11:20 -07:00
Xiaoxi Wang
a3d0b005dc reset several method use getShardMetrics 2022-05-04 00:00:03 -07:00
Xiaoxi Wang
7c37d172b9 solve some comments 2022-05-03 17:21:08 -07:00
Xiaoxi Wang
00b97ec829 add storage metric compare knob; timeThrottle with constant 2022-04-27 23:37:35 -07:00
Xiaoxi Wang
73c7240bfd code format 2022-04-26 17:34:34 -07:00
Xiaoxi Wang
9b211c48ee format code 2022-04-26 17:13:37 -07:00
Xiaoxi Wang
8cc6d9aaac determine timeThrottle and topK dynamically on the shard number 2022-04-25 16:59:20 -07:00
Xiaoxi Wang
cc05f5e9db fix getMetrics keys bug 2022-04-24 21:21:18 -07:00
Xiaoxi Wang
a0b45c2962 fix throttle bug 2022-04-24 15:51:18 -07:00
Xiaoxi Wang
eefd0778c5 change canQueue to timeThrottle() 2022-04-22 15:26:44 -07:00
Xiaoxi Wang
4818f01de9 fix top10 shard index bug; add event detail; fix merge conflict 2022-04-22 14:14:58 -07:00
Xiaoxi Wang
0639810b66 merge upstream/main 2022-04-22 11:09:15 -07:00
Xiaoxi Wang
eb949ee3dc change default shard size; enable valley filler best; top 10 random choice 2022-04-22 10:07:12 -07:00
Xiaoxi Wang
04311d001e topK shard random selection 2022-04-21 22:37:16 -07:00
Xiaoxi Wang
960a13f233 format 2022-04-21 21:40:49 -07:00
Xiaoxi Wang
f9e57396e2 completeDest 2022-04-21 21:23:42 -07:00
Xiaoxi Wang
a8bfd0f481 move canQueue, 60s each source server, random select portion of shards/destServer 2022-04-20 23:12:34 -07:00
Xiaoxi Wang
af9e5ba885 move canQueue, 60s each source server, random select portion of shards 2022-04-20 22:19:56 -07:00
Xiaoxi Wang
131adec811 add canQueue 2022-04-20 15:28:03 -07:00
Xiaoxi Wang
25e1e75d9e enable destComplete 2022-04-20 13:32:04 -07:00
Xiaoxi Wang
e0ae0942c4 fix inflight read division; temp destComplete fix; 0.1 constant poll time 2022-04-20 12:15:40 -07:00
Xiaoxi Wang
a7a9de781e update unittests 2022-04-16 22:51:55 -07:00
Xiaoxi Wang
7b9432bdda revert to original BgDDValleyFiller and BgDDMountainChopper for disk rebalance 2022-04-12 18:01:28 -07:00
Xiaoxi Wang
8d3f851495 merge upstream/mainA 2022-04-12 17:03:09 -07:00
Xiaoxi Wang
ed97a35dc0 Merge branch 'main' into readaware 2022-04-12 16:47:15 -07:00
Xiaoxi Wang
5e96bacb5b add new priority in RelocateData 2022-04-12 16:22:17 -07:00
Xiaoxi Wang
61a1f7683b fix dd command line read special key space error 2022-04-11 22:49:21 -07:00
Xiaoxi Wang
6c841cd32b merge readaware-better 2022-04-11 17:09:39 -07:00
Bharadwaj V.R
129a7b5daf Use boolean-param for GetTeamRequest params 2022-04-11 13:27:08 -07:00
Xiaoxi Wang
411550580e merge 2022-04-08 11:49:33 -07:00
Xiaoxi Wang
7c83f63e46 fix substraction bug; test1,2,3 generate good result 2022-04-08 10:44:56 -07:00
sfc-gh-tclinkenbeard
e27b0d9ab5 Merge remote-tracking branch 'origin/main' into improve-snapshot-fault-tolerance 2022-04-07 23:30:16 -07:00
sfc-gh-tclinkenbeard
91930b8040 Remove getMinReplicasRemaining PromiseStream.
Instead, in order to enforce the maximum fault tolerance for snapshots,
update getStorageWorkers to return the number of unavailable storage
servers (instead of throwing an error when unavailable storage servers
exist).
2022-04-07 23:23:23 -07:00
Xiaoxi Wang
c858b0eaa9 update penalty, reset sim skip to 3; add read_underutil to relocator priority 2022-04-07 11:59:40 -07:00
Xiaoxi Wang
6fa60a2cf9 fix substraction typo 2022-04-06 23:03:25 -07:00
Xiaoxi Wang
150b4318aa refactor datadistribution command; try dual-mode code 2022-04-06 22:10:23 -07:00
Xiaoxi Wang
aba9d85560 merge main 2022-04-06 09:57:52 -07:00
Xiaoxi Wang
b6594aebbf change inflight substraction priority (not non-negligible performance influence 2022-04-04 11:34:41 -07:00
sfc-gh-tclinkenbeard
70f378bacc Restrict write access to getUnhealthyRelocationCount 2022-04-03 23:47:54 -07:00
sfc-gh-tclinkenbeard
253db642be Add MAX_SNAPSHOT_FAULT_TOLERANCE knob 2022-04-03 22:31:45 -07:00
Josh Slocum
9e06881673 fix destination limiting and cancelling logic in move_to_removed_server case 2022-03-31 14:05:15 -05:00