200 Commits

Author SHA1 Message Date
Jingyu Zhou
cfcf0f152c Merge branch 'main-4a085fc84' into vv
Fix Conflicts:
	fdbclient/NativeAPI.actor.cpp
	fdbserver/ClusterRecovery.actor.cpp
	fdbserver/MasterInterface.h
	fdbserver/masterserver.actor.cpp
	flow/error_definitions.h
2022-03-30 22:28:06 -07:00
sfc-gh-tclinkenbeard
a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
Sreenath Bodagala
72e06369a4 Merge remote-tracking branch 'apple-upstream/main' into version-vector-prototype 2022-02-08 17:47:57 +00:00
Jingyu Zhou
f361f5a96c Rename isEmptyMessage to messagesWritten
The variable name means exactly the opposite.
2022-02-03 14:55:44 -08:00
Dan Lambright
9544379cdf rebase 2022-01-20 11:12:33 -05:00
Dan Lambright
9f4ac866cd Avoid context switch between appending version list and updating dv
Port PR 6117 (Resolver saves shardChanged in recent state transactions)
2021-12-13 13:02:32 -05:00
Dan Lambright
0222d8669d fix simulation failures 2021-12-10 09:56:21 -05:00
Lukas Joswiak
30867750b5 Add protection against storage and tlog data deletion when joining a new cluster 2021-11-09 12:29:47 -08:00
Jingyu Zhou
c1d7b03087 CommitProxy uses Resolver's private mutations
Note the commit proxy still runs applyMetadataMutations for setting up memory
states, and no longer generates private mutations.
2021-10-15 09:47:23 -04:00
Jingyu Zhou
9ecdbc39ee Add private mutations to Resolver reply messages 2021-10-15 09:45:57 -04:00
Dan Lambright
58e1888d8e remove network hop by getting previous commit versions in GetCommitVersionRequest 2021-09-30 11:51:57 -04:00
Sreenath Bodagala
184c134b8a - Resolve merge conflicts 2021-09-17 20:25:16 +00:00
Sreenath Bodagala
2aa3b44d4e Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype
- Conflicts:
	fdbserver/LogSystem.h
	fdbserver/LogSystemConfig.h
	fdbserver/TagPartitionedLogSystem.actor.cpp

- Files modified during merge:

modified:   fdbserver/LogSystem.cpp
modified:   fdbserver/LogSystemConfig.cpp
2021-09-17 19:36:18 +00:00
Sreenath Bodagala
6716ed0d17 Address build failure. 2021-09-16 12:00:26 -04:00
Xiaoge Su
067c1cc55b Extract methods in LogSystem.h to corresponding cpp file 2021-09-12 14:17:19 -07:00
Dan Lambright
8689e1f106 merge with master 2021-08-30 15:29:08 -04:00
FDB Formatster
2c788c233d apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-08-27 17:07:47 -07:00
Xiaoxi Wang
1f6cee89ab merge master, fix conflicts 2021-08-10 10:01:45 -07:00
Sreenath Bodagala
a081c0baa5 Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype 2021-08-05 22:40:32 +00:00
yao-xiao-github
8609b45354
Add histograms to CommitProxyServer. (#5299) 2021-08-05 09:17:37 -07:00
Xiaoxi Wang
2df0474fec merge master 2021-08-02 11:58:35 -07:00
Xiaoxi Wang
10c82b422f merge master branch 2021-07-28 14:19:46 -07:00
sfc-gh-tclinkenbeard
c74047c665 Merge remote-tracking branch 'origin/master' into fix-more-clang-warnings 2021-07-28 11:51:02 -07:00
Steve Atherton
507c1f11e3 Add .log() to bare TraceEvent() invocations without any .detail()s to avoid clang-tidy warning about immediate destruction of object without use. 2021-07-26 19:55:10 -07:00
Sreenath Bodagala
82774ae62c - Address review comments 2021-07-26 19:39:08 +00:00
Sreenath Bodagala
de020efa7f - Correct a comment 2021-07-26 17:13:31 +00:00
Sreenath Bodagala
d398377c4a - Move the logic that populates written_tags (the tags that have been
affected by a transaction batch) to LogPushData
2021-07-26 16:27:19 +00:00
sfc-gh-tclinkenbeard
b9a22a61ef Fix many -Wreorder-ctor warnings 2021-07-23 17:33:18 -07:00
Dan Lambright
d07c8ce211 wait for prev per tlog 2021-07-15 08:44:48 -04:00
Dan Lambright
17be3dc201 fix bug related to misunderstanding of tag replicas. 2021-07-12 11:46:52 -04:00
Dan Lambright
a107dd655e tpcv messaging 2021-07-12 11:46:52 -04:00
Xiaoxi Wang
569de5259f Merge branch 'master' of https://github.com/apple/foundationdb into tlog_dev 2021-07-06 23:15:20 +00:00
Xiaoxi Wang
b6d5c8a091 implement tLogPeekStream 2021-07-06 23:14:58 +00:00
Xiaoxi Wang
b50fda6b4b add simple streaming peek functions 2021-07-01 23:17:28 +00:00
Jingyu Zhou
9c82b22aa3 Add comments for recordEmptyMessage and getEmptyMessageRatio. 2021-06-28 16:09:07 -07:00
Jingyu Zhou
c700feaa6e Address Dan's comments 2021-06-28 11:16:35 -07:00
Jingyu Zhou
66a97b76f1 Add metric to track the ratio of empty messages to tlogs
This metric is to understand the impact of batching on empty commit messages.
If the ratio is high, it means proxy sends many empty commit messages, which
can potentially be optimized for better performance. If the ratio is low, which
means the spread factor is large, thus we need to optimize the proxy to reduce
such a factor.
2021-06-24 21:26:09 -07:00
Sreenath Bodagala
b0554b4554 Capture how fast an SS is catching up to its tLog-SS lag
Changes:
LogSystem.h, LogSystemPeekCursor.actor.cpp:
Add APIs to find the ID of the tLog from which an SS has fetched the latest
set of versions.

storageserver.actor.cpp:
Capture the number of latest set of versions fetched, the time (in seconds)
in which those versions were fetched, and the tLog from which they were
fetched. Add this information to a TraceLogEvent.

Capture how many versions an SS has fetched in the
2021-05-11 20:03:21 +00:00
FDB Formatster
df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Evan Tschannen
346a4e3ecd Merge branch 'release-6.3'
# Conflicts:
#	fdbcli/fdbcli.actor.cpp
#	fdbrpc/LoadBalance.actor.h
#	fdbrpc/MultiInterface.h
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/masterserver.actor.cpp
2021-03-01 18:52:06 -08:00
Meng Xu
33eb1de00e Add some comment to log system
and resolve review comment by deleting my questions.
2021-02-19 21:44:13 -08:00
sfc-gh-tclinkenbeard
5bfa6cea98 Merge remote-tracking branch 'origin/master' into misc-changes 2020-12-26 20:47:00 -04:00
sfc-gh-tclinkenbeard
5020e3faa1 Make ILogSystem::IPeekCursor const-correct 2020-12-08 09:09:31 -08:00
sfc-gh-tclinkenbeard
dd3669886e Improve IPeekCursor and ILogSystem method signatures 2020-12-08 09:09:31 -08:00
Richard Chen
c77d9e4abe merge conflicts 2020-12-02 21:53:19 +00:00
Lukas Joswiak
7442e43dbf Add TEST and ASSERT macros 2020-10-30 13:41:56 -07:00
Richard Chen
bf7bf470d9 Merge branch 'master' of https://github.com/apple/foundationdb into stable_interfaces 2020-10-26 19:36:19 +00:00
Lukas Joswiak
22bc2eb71c Fix subsequence ordering issue 2020-10-26 01:15:26 -07:00
Lukas Joswiak
7403e49c33 Fix deserialization issue with late LogProtocolMessage
Previously, if multiple metadata messages were written without clearing
transaction state between them, an issue could occur when deserializing
where a later message could be missing its LogProtocolMessage prefix.
This would cause the protocol version to not be set, causing an
assertion failure when deserializing MutationRefs.
2020-10-26 01:13:51 -07:00
Lukas Joswiak
09721ba9bf Only send span context info if supported 2020-10-23 16:36:05 -07:00