apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-05-18 03:42:10 +08:00
Code Issues Packages Projects Releases Wiki Activity
12,362 Commits 35 Branches 318 Tags
Commit Graph

141 Commits

Author SHA1 Message Date
FDB Formatster
8a8c488ede apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-05 18:13:38 -06:00
Andrew Noyes
dc2bac5670 Resolve conflicts 2020-11-24 19:09:42 +00:00
Andrew Noyes
1f541f02be Merge branch 'anoyes/merge-6.2-to-6.3' into anoyes/release-6.3-merge 
Merge, leaving conflict markers for now
 2020-11-24 16:55:34 +00:00
David Youngworth
489ba20641 Fix several merge issues 2020-11-16 14:46:36 -08:00
David Youngworth
d0391db862 Merge branch 'release-6.2' into release-6.3 2020-11-16 10:15:23 -08:00
Xiaoge Su
15d10d68cd Cleanup DataDistributionQueue.actor.cpp and storageserver.actor.cpp 2020-11-09 15:23:51 -08:00
Xin Dong
2909e35415 When moving dispaching shard relocations, log the source and destination storage servers. 2020-11-05 16:13:18 -08:00
Xin Dong
cedcfc7d87 Removed unused uid field inside ParallelTCInfo 2020-10-26 16:48:09 -07:00
Xin Dong
9ef29d0cea Changed getTeamID() to return a string instead of UID as suggested by reviews. 2020-10-26 16:44:52 -07:00
Xin Dong
7ebb2e5c09 Piggy back this PR to polish more TraceEvent by: 
- Making it clear that it's tracking machine team info or server team info
- Added ID to both machine team and server team for better trackability
- Attach distributor id to some trace events.
 2020-10-26 16:44:09 -07:00
A.J. Beamon
3b66a1f2d4 Fix a couple places where we were creating vectors with default elements rather than reserving space. 2020-10-09 10:51:06 -07:00
sfc-gh-tclinkenbeard
83c5a30f62 Add encapsulation to TCTeamInfo and ParallelTCInfo 2020-07-21 11:05:41 -07:00
sfc-gh-tclinkenbeard
9a2ce4c981 Make IDataDistributionTeam const-correct 2020-07-21 11:05:34 -07:00
Meng Xu
815e403009 Add comment to DDQueue GetTeam 
Comments to help understand Evan's PR 3487
that fix the problem: replication factor could drop unexpected in fearless config.
 2020-07-13 17:05:50 -07:00
A.J. Beamon
b09dddc07e Merge branch 'release-6.2' into merge-release-6.2-into-release-6.3 
# Conflicts:
#	cmake/ConfigureCompiler.cmake
#	documentation/sphinx/source/downloads.rst
#	fdbrpc/FlowTransport.actor.cpp
#	fdbrpc/fdbrpc.vcxproj
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/Knobs.cpp
#	fdbserver/Knobs.h
#	fdbserver/LogSystemPeekCursor.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/Status.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	flow/flow.vcxproj
 2020-07-10 15:06:34 -07:00
Evan Tschannen
0e2f5e8bb5 Added a flow lock to prevent too many source server fetches from happening at the same time and running the data distributor out of memory 2020-07-09 10:38:19 -07:00
Evan Tschannen
5e02fd490e fix: the check for if a teamCollection was tracking a source server was unreliable, leading to scenarios where we would temporarily replicate a shard less than teamSIze 2020-06-29 10:02:27 -07:00
Evan Tschannen
48b1b20f67 Fixed a crash related to destruction order in data distribution 2020-05-10 23:14:19 -07:00
A.J. Beamon
b1172417f5 Merge branch 'master' into per-priority-busy-logging 
# Conflicts:
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/Net2.actor.cpp
 2020-04-14 14:22:12 -07:00
Alex Miller
1439de37b5 Convert GetRangeLimits() -> TOO_MANY + ASSERT(). 2020-04-12 18:23:14 -07:00
Alex Miller
6078fd1b18 Convert UID to Tag in keyServers to reduce txnStateStore size 2020-04-05 14:30:09 -07:00
Evan Tschannen
e08f0201f1 merge release 6.2 into master 2020-03-17 12:51:47 -07:00
Evan Tschannen
12f2b32770 added additional logging in data distribution 2020-03-13 15:19:33 -07:00
Evan Tschannen
303df197cf Merge branch 'release-6.2' 
# Conflicts:
#	CMakeLists.txt
#	bindings/c/test/mako/mako.c
#	documentation/sphinx/source/release-notes.rst
#	fdbbackup/backup.actor.cpp
#	fdbclient/NativeAPI.actor.cpp
#	fdbclient/NativeAPI.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/Knobs.cpp
#	fdbserver/Knobs.h
#	fdbserver/LogRouter.actor.cpp
#	fdbserver/SkipList.cpp
#	fdbserver/fdbserver.actor.cpp
#	flow/CMakeLists.txt
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/flow.vcxproj
#	flow/flow.vcxproj.filters
#	versions.target
 2020-03-06 18:22:46 -08:00
Evan Tschannen
e219c1671f Merge branch 'release-6.2' into feature-dd-region-queue 
# Conflicts:
#	fdbserver/Knobs.h
 2020-03-04 16:25:38 -08:00
Evan Tschannen
6d6f184e2f added a knob which reverts the new queue behavior 2020-03-04 16:23:49 -08:00
Evan Tschannen
b7834b2995
Merge pull request #2774 from etschannen/feature-dd-repopulate-priority 
Make the DD priority of populating a region lower than machine failures
 2020-03-04 16:15:18 -08:00
Evan Tschannen
125bd13198 fix: in multi-region configurations, the data distribution queue could start too much work, expecting that the remote region would contribute to the read workload 2020-03-04 14:17:17 -08:00
Evan Tschannen
6296465e07 Make the DD priority associated with populating a remote region lower than machine failures 2020-03-04 14:07:32 -08:00
Meng Xu
1ef4cb432b Merge branch 'master' into mengxu/fast-restore-robust-and-visibility-PR-v2 2020-03-01 20:08:07 -08:00
Meng Xu
ad9b3fb4a8 DD:Add trace for detailed relocate shard info 2020-02-29 13:45:10 -08:00
Meng Xu
2657d41bb2 FastRestore:Add debug msg when memory is over threshold 2020-02-27 18:32:11 -08:00
Alvin Moore
0f64505d0b Merge branch 'release-6.2' of github.com:apple/foundationdb 
Needed to pull in changes to build docker
 2020-02-23 23:27:53 -08:00
Evan Tschannen
96258b9809 Merge branch 'release-6.2' 
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbcli/fdbcli.actor.cpp
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/FlowTransport.actor.cpp
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/DataDistribution.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/QuietDatabase.actor.cpp
#	fdbserver/SkipList.cpp
#	fdbserver/StorageMetrics.actor.h
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/KVStoreTest.actor.cpp
#	flow/CMakeLists.txt
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/genericactors.actor.cpp
#	flow/serialize.h
 2020-02-21 19:09:16 -08:00
A.J. Beamon
4c696d5bf2 Merge branch 'release-6.2' into dd-better-rebalance-logging 
# Conflicts:
#	fdbserver/DataDistributionQueue.actor.cpp
 2020-02-21 17:41:00 -08:00
A.J. Beamon
dfa5f76c01 Remove unused parameter. Don't put check for g_network presence in ASSERT_WE_THINK. 2020-02-21 16:28:03 -08:00
Evan Tschannen
08914a2acd Once available space ratio falls below 0.3 avoid moving data to teams with less free space than the median team 2020-02-21 15:14:32 -08:00
A.J. Beamon
2e699fef55 Don't suppress actor cancellation because we've already initialized the trace event by adding details. 2020-02-21 11:28:59 -08:00
A.J. Beamon
6810a03283 Add more logging to valley filler and mountain chopper 2020-02-21 10:55:14 -08:00
Evan Tschannen
819c55556c More aggressively attempt to find teams that do not have low disk space 2020-02-20 16:47:50 -08:00
A.J. Beamon
e1fb568fd1 Merge branch 'release-6.2' into dd-use-available-space 
# Conflicts:
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/DataDistribution.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
 2020-02-20 16:12:42 -08:00
A.J. Beamon
e4b483796d Combine some logic that was doing similar computations for free space ratio. 2020-02-20 14:52:08 -08:00
A.J. Beamon
4c9c736253 Data distribution uses available space instead of free space when evaluating whether processes are low on space and penalizing them. 2020-02-20 11:21:03 -08:00
A.J. Beamon
c164acb88d Add new criteria to DD's GetTeamRequest that allow you to require shards be present on the team and that the team have a minimum free ratio. This avoids scenarios where the team chosen when processing the request is later rejected by the requestor, causing rebalancing movements to get stuck. 2020-02-20 09:32:00 -08:00
A.J. Beamon
abb75f7eb7 Add logging to indicate the time spent at each priority that exceeds some minimum busyness threshold 2020-02-07 14:34:24 -08:00
Evan Tschannen
3f9d9d8b84 Merge branch 'release-6.2' 
# Conflicts:
#	CMakeLists.txt
#	cmake/FlowCommands.cmake
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/StorageServerInterface.h
#	fdbserver/DataDistributionTracker.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	flow/Knobs.h
#	flow/Platform.cpp
#	versions.target
 2020-01-16 18:37:47 -08:00
Evan Tschannen
e65760eb46
Merge pull request #2536 from etschannen/feature-commit-latency 
Improved commit latency in large clusters
 2020-01-13 19:12:02 -08:00
Evan Tschannen
ab7071932f Data distribution no longer attempts to pick teams which share members of the source unless the team matches exactly 2020-01-09 16:59:37 -08:00
Evan Tschannen
3157d8a375 fixed typo 2019-12-18 16:57:39 -08:00
Evan Tschannen
688940b685 merge 6.2 into master 2019-10-21 11:43:46 -07:00