He Liu
fa418fd784
Change SHARD_ENCODE_LOCATION_METADATA to a server knob. ( #7770 )
...
Co-authored-by: He Liu <heliu@apple.com>
2022-08-03 13:51:40 -07:00
Xiaoxi Wang
f0caaad93c
Update fdbserver/DataDistributionQueue.actor.cpp
2022-07-29 10:22:21 -07:00
Xiaoxi Wang
e3cf40002d
format code
2022-07-29 10:11:10 -07:00
Xiaoxi Wang
d8323820f1
remove duplicate code
2022-07-29 10:04:14 -07:00
Bharadwaj V.R
dcf09fd691
Move DD queue code over to using movement-reasons rather than priority ( #7614 )
...
* Use enum variables to invoke Priority checking
* add an explicit isDataMovementForReadBalancing function
* Set up RelocateShard in terms of data movement reason instead of priority
* Remove isMountainChopperPriority
* Remove isDiskRebalancePriority
* Fix formatting
* Fix misnamed DataMovementReason::TEAM_HEALTHY
Co-authored-by: Zhongxing Zhang <zhongxing.zhang@snowflake.com>
2022-07-25 00:50:37 -07:00
Xiaoxi Wang
7b6f1ca712
Merge pull request #7621 from sfc-gh-xwang/feature/dd-refactor-incremental
...
[DD Testability] Add unittest for resuming infligh relocations
2022-07-23 22:14:45 -07:00
Xiaoxi Wang
cd13356964
solve review comments
2022-07-20 16:09:38 -07:00
Xiaoxi Wang
43099bda0d
resolve conflict upstream/main
2022-07-19 13:48:39 -07:00
Xiaoxi Wang
7865ead54d
Merge branch 'main' into readaware
2022-07-19 13:35:33 -07:00
Markus Pilman
1de37afd52
Make TEST macros C++ only ( #7558 )
...
* proof of concept
* use code-probe instead of test
* code probe working on gcc
* code probe implemented
* renamed TestProbe to CodeProbe
* fixed refactoring typo
* support filtered output
* print probes at end of simulation
* fix missed probes print
* fix deduplication
* Fix refactoring issues
* revert bad refactor
* make sure file paths are relative
* fix more wrong refactor changes
2022-07-19 13:15:51 -07:00
Xiaoxi Wang
2909614b0c
smaller function
2022-07-18 14:21:50 -07:00
Zhongxing Zhang
9c20b15f5a
Use enum variables to invoke Priority checking ( #7514 )
...
* Use enum variables to invoke Priority checking
* add an explicit isDataMovementForReadBalancing function
2022-07-14 09:06:56 -07:00
Xiaoxi Wang
f5a81154ef
move takeMoveKeysLock to DDTxnProcessor
2022-07-08 14:11:31 -07:00
He Liu
bc5bfaffda
Shard based move ( #6981 )
...
* Shard based move.
* Clean up.
* Clear results on retry in getInitialDataDistribution.
* Remove assertion on SHARD_ENCODE_LOCATION_METADATA for compatibility.
* Resolved comments.
Co-authored-by: He Liu <heliu@apple.com>
2022-07-07 20:49:16 -07:00
Xiaoxi Wang
ab2f2bc9b3
format code
2022-06-23 09:16:42 -07:00
Xiaoxi Wang
c9ac30612f
Merge branch 'main' of https://github.com/apple/foundationdb into readaware
2022-06-23 09:12:07 -07:00
Xiaoxi Wang
ef0f415e3d
add option; change to shared_ptr
2022-06-13 13:55:48 -07:00
Xiaoxi Wang
1de6c09307
use struct instead of tuple
2022-06-13 11:27:50 -07:00
Xiaoxi Wang
c12a7a30ed
Update fdbserver/DataDistributionQueue.actor.cpp
...
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
2022-06-13 08:22:48 -07:00
Xiaoxi Wang
b99bd45730
format code
2022-06-09 12:36:20 -07:00
Xiaoxi Wang
178bab5faf
Merge branch 'readaware-perf' into readaware
2022-06-09 12:23:41 -07:00
Xiaoxi Wang
e5aa5fef22
merge upstream/main
2022-06-09 12:17:27 -07:00
Xiaoxi Wang
6ab12ea971
add storeTuple and unit test; refactor getSourceServersForRange
2022-06-09 12:16:12 -07:00
Bharadwaj V.R
24b2be1c4f
Remove last-limited check from DDMountainChopper and DDValleyFiller
2022-05-30 21:57:34 -07:00
Xiaoxi Wang
9b8859f4f2
move keys out of StorageMetrics
2022-05-27 17:10:01 -07:00
Xiaoxi Wang
4fb2e90be9
add comment
2022-05-27 12:14:34 -07:00
Xiaoxi Wang
74748c20a0
a.fix heap-use-after-free caused by early noErrorsActors destroy
2022-05-27 10:14:10 -07:00
Xiaoxi Wang
85d1b59172
fix heap-use-after-free caused by early noErrorsActors destroy
2022-05-26 15:55:14 -07:00
Xiaoxi Wang
13a77dd5a2
change priority knob; change PromiseStream to FutureStream; remove comments; add on_sr check
2022-05-25 17:09:34 -07:00
Xiaoxi Wang
73624bcd2a
Merge remote-tracking branch 'upstream/main' into readaware
2022-05-23 11:17:38 -07:00
Xiaoxi Wang
54e56af3db
solve some review comments
2022-05-23 11:04:37 -07:00
Xiaoxi Wang
3fee0f33b5
change reference pointer
2022-05-23 00:12:48 -07:00
Xiaoxi Wang
27a60df36d
fix rename bug
2022-05-22 23:37:32 -07:00
Xiaoxi Wang
0579b577ba
remove polling interval; uncomment suppressFor
2022-05-22 23:35:39 -07:00
Xiaoxi Wang
4f28af1ab7
add comment
2022-05-19 10:50:24 -07:00
Xiaoxi Wang
8950822e36
add knobs; change knobs
2022-05-17 14:49:27 -07:00
Xiaoxi Wang
f40a2a68fc
add more statistics
2022-05-17 10:19:09 -07:00
Xiaoxi Wang
3b241955e7
add more informative trace info
2022-05-16 21:25:56 -07:00
Xiaoxi Wang
dc5d2bff06
make ratekeeper don't limit read rebalance
2022-05-16 11:38:38 -07:00
Bharadwaj V.R
30922268d7
Minor readability improvement (I believe) to DD code; mostly, replacing .first, .second of pairs with better names through bindings)
2022-05-13 13:03:00 -07:00
Xiaoxi Wang
e9e11bf53b
change all criteria to knobs
2022-05-12 16:30:21 -07:00
Xiaoxi Wang
73e655a067
adjust (srcLoad - destLoad) / 10.0
2022-05-09 16:11:38 -07:00
Xiaoxi Wang
b0a00effa0
set max shard bandwidth
2022-05-06 16:37:12 -07:00
Xiaoxi Wang
03607675e8
refactor GetTeamRequest
2022-05-04 17:42:49 -07:00
Xiaoxi Wang
964e0cecef
format
2022-05-04 14:52:39 -07:00
Xiaoxi Wang
22eafcf7a2
rename trace event
2022-05-04 14:27:42 -07:00
Xiaoxi Wang
ae66ed6c16
fix DataDistributionQueue time_out ; reset the rebalance poll time
2022-05-04 14:11:20 -07:00
Xiaoxi Wang
a3d0b005dc
reset several method use getShardMetrics
2022-05-04 00:00:03 -07:00
Xiaoxi Wang
7c37d172b9
solve some comments
2022-05-03 17:21:08 -07:00
Xiaoxi Wang
00b97ec829
add storage metric compare knob; timeThrottle with constant
2022-04-27 23:37:35 -07:00
