add informal classification of priority

design/data-distributor-internals.md

@@ -82,7 +82,13 @@ Actors are created to monitor the reasons of key movement:
 (4) `teamTracker` actor monitors a team’s healthiness. When a server in the team becomes unhealthy, it issues the `RelocateShard` request to repair the replication factor. The less servers a team has, the higher priority the `RelocateShard` request will be.
 
 #### Movement Priority
-Each shard movement has a priority associating with the move attempt, which is depended on the priority of the source team. The explanation of each priority knob (`PRIORITY_<XXX>`) is in `ServerKnobs.h`.
+There are roughly 4 class of movement priorities
+* Healthy priority. The movement is for maintain the cluster healthy status, and the priority is depended on the healthy status of the source team.
+* Load balance priority. The movement is for balance cluster workload.
+* Boundary change priority. The movement will change current shard boundaries.
+* Others. Like resuming a in-flight movement.
+
+Each shard movement has a priority associating with the move attempt,  The explanation of each priority knob (`PRIORITY_<XXX>`) is in `ServerKnobs.h`.
 
 In `status json` output, please look at field `.data.team_tracker.state` for team priority state.
 

fdbclient/include/fdbclient/ServerKnobs.h

@@ -143,28 +143,47 @@ public:
 	//   mean better worst case time bounds
 	// Maximum allowable priority is 999.
 	// Update the status json .data.team_tracker.state field when necessary
-	int PRIORITY_RECOVER_MOVE; // Priority for movement resume from previous unfinished in-flight movement when a new DD
+	//
+	// Priority for movement resume from previous unfinished in-flight movement when a new DD
 	// start
-	int PRIORITY_REBALANCE_UNDERUTILIZED_TEAM; // For disk valley filler
-	int PRIORITY_REBALANCE_OVERUTILIZED_TEAM; // For disk mountain chopper
-	int PRIORITY_REBALANCE_READ_OVERUTIL_TEAM; // For read valley filler
-	int PRIORITY_REBALANCE_READ_UNDERUTIL_TEAM; // For read mountain chopper
-	int PRIORITY_PERPETUAL_STORAGE_WIGGLE; // Priority for wiggle a storage server
-	int PRIORITY_TEAM_HEALTHY; // Priority when all servers in a team are healthy
-	int PRIORITY_TEAM_CONTAINS_UNDESIRED_SERVER; // Priority when there's undesired servers in the team. (ex. same ip
+	int PRIORITY_RECOVER_MOVE;
+	// A load-balance priority for disk valley filler
+	int PRIORITY_REBALANCE_UNDERUTILIZED_TEAM;
+	// A load-balance priority disk mountain chopper
+	int PRIORITY_REBALANCE_OVERUTILIZED_TEAM;
+	// A load-balance priority read valley filler
+	int PRIORITY_REBALANCE_READ_OVERUTIL_TEAM;
+	// A load-balance priority read mountain chopper
+	int PRIORITY_REBALANCE_READ_UNDERUTIL_TEAM;
+	// A team healthy priority for wiggle a storage server
+	int PRIORITY_PERPETUAL_STORAGE_WIGGLE;
+	// A team healthy priority when all servers in a team are healthy
+	int PRIORITY_TEAM_HEALTHY;
+	// A team healthy priority when there's undesired servers in the team. (ex. same ip
 	// address as other SS process, or SS is lagging too far ...)
-	int PRIORITY_TEAM_REDUNDANT; // Priority for removing redundant team to make the team count within a good range
+	int PRIORITY_TEAM_CONTAINS_UNDESIRED_SERVER;
+	// A team healthy priority for removing redundant team to make the team count within a good range
+	int PRIORITY_TEAM_REDUNDANT;
+	// A shard boundary priority for merge small and write cold shard.
 	int PRIORITY_MERGE_SHARD;
-	int PRIORITY_POPULATE_REGION; // Priority for populate remote region
-	int PRIORITY_TEAM_UNHEALTHY; // Priority when the replica > 3 and there's at least one unhealthy server in a team.
+	// A team healthy priority for populate remote region
+	int PRIORITY_POPULATE_REGION;
+	// A team healthy priority when the replica > 3 and there's at least one unhealthy server in a team.
 	// Or when the team contains a server with wrong configuration (ex. storage engine,
 	// locality, excluded ...)
-	int PRIORITY_TEAM_2_LEFT; // Priority when the replica <= 3 and there's 2 healthy servers in a team
-	int PRIORITY_TEAM_1_LEFT; // Priority when the replica <= 3 and there's 1 healthy server in a team
-	int PRIORITY_TEAM_FAILED; // Priority when a server in the team is excluded as failed
-	int PRIORITY_TEAM_0_LEFT; // Priority when the replica <= 3 and there's no healthy server in a team
+	int PRIORITY_TEAM_UNHEALTHY;
+	// A team healthy priority when the replica <= 3 and there's 2 healthy servers in a team
+	int PRIORITY_TEAM_2_LEFT;
+	// A team healthy priority when the replica <= 3 and there's 1 healthy server in a team
+	int PRIORITY_TEAM_1_LEFT;
+	// A team healthy priority when a server in the team is excluded as failed
+	int PRIORITY_TEAM_FAILED;
+	// A team healthy priority when the replica <= 3 and there's no healthy server in a team
+	int PRIORITY_TEAM_0_LEFT;
+	// A shard boundary priority for split large or write hot shard.
 	int PRIORITY_SPLIT_SHARD;
-	int PRIORITY_ENFORCE_MOVE_OUT_OF_PHYSICAL_SHARD; // Priority when a physical shard is oversize or anonymous
+	// Priority when a physical shard is oversize or anonymous
+	int PRIORITY_ENFORCE_MOVE_OUT_OF_PHYSICAL_SHARD;
 
 	// Data distribution
 	bool SHARD_ENCODE_LOCATION_METADATA; // If true, location metadata will contain shard ID.