apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-06-01 10:45:56 +08:00
Code Issues Packages Projects Releases Wiki Activity
15,995 Commits 35 Branches 318 Tags
Commit Graph

127 Commits

Author SHA1 Message Date
Evan Tschannen
b495cc697b Merge branch 'release-6.2' 
# Conflicts:
#	CMakeLists.txt
#	documentation/sphinx/source/release-notes.rst
#	versions.target
 2019-09-13 09:25:08 -07:00
Alex Miller
324289039a When reloading one cursor in a merge cursor, top off the other cursors as well. 2019-09-12 16:22:28 -07:00
Jingyu Zhou
2723922f5f Replace -1 as VERSION_HEADER constant for serialization 2019-09-05 12:45:39 -07:00
Jingyu Zhou
f9357c5ad8 Fix side effect of ArenaReader 
ServerPeekCursor::nextMessage() should only consume the message header, because
the reader() directly inherits the current position. The previous commit
changes the positon to the begining of the next message, which breaks storage
server code.
 2019-09-05 11:07:07 -07:00
Jingyu Zhou
cd3f1e33d4 Refactor deserialization of TagsAndMessages 
Consolidate deserialization of TagsAndMessages in the structure itself and
change both TLog and ServerPeekCursor to use it.
 2019-09-04 14:55:05 -07:00
Evan Tschannen
b0480edd15 fix: messageVersion could be larger than poppedVersion, and we will discard messages that are needed 2019-08-06 16:31:05 -07:00
Evan Tschannen
7ac7eb82f2 fix: buffered cursor would start multiple bufferedGetMore actors 
advance all of the cursors to the poppedVersion
 2019-07-30 14:42:05 -07:00
Evan Tschannen
b5cb7919b6 fix: canDiscardPopped was not reset when necessary in all cases 2019-07-30 13:44:44 -07:00
Evan Tschannen
5d79e4141f fix: buffered cursor messageVersion should be set to the version we will be at after exhausting everything in messages 2019-07-30 12:38:44 -07:00
Evan Tschannen
6977e7d2e8 do not return recovered version as popped for txsTags because it could cause recovery to start over 
optimized how buffered peek cursor discards popped data
 2019-07-30 12:21:48 -07:00
Evan Tschannen
7a932479dd throw away state if we ever read popped data from the disk queue adapter 2019-07-30 10:14:39 -07:00
Evan Tschannen
45f7b41b48 fix: multi-cursor could discard popped commits after already returning data 2019-07-29 21:36:42 -07:00
Evan Tschannen
5bb322b483 implement popped on bufferedCursor 2019-07-29 21:19:47 -07:00
Evan Tschannen
d8948c8be1 Merge branch 'master' into feature-fast-txs-recovery 
# Conflicts:
#	fdbserver/TagPartitionedLogSystem.actor.cpp
 2019-07-10 13:59:52 -07:00
Evan Tschannen
b27a909f3a fix: onDisconnectOrFailure can spuriously trigger 2019-07-09 16:38:59 -07:00
Evan Tschannen
15e894c724 Merge in master 2019-07-05 15:49:24 -07:00
Evan Tschannen
cfce1e1705 fix: buffered peek cursor would advance very slowly through large ranges of empty versions 2019-06-28 15:54:08 -07:00
Alex Miller
bf883d7055 Merge remote-tracking branch 'upstream/master' into flowlock-api 2019-06-25 14:26:50 -07:00
Alex Miller
7a500cd37f A giant translation of TaskFooPriority -> TaskPriority::Foo 
This is so that APIs that take priorities don't take ints, which are
common and easy to accidentally pass the wrong thing.
 2019-06-25 02:47:35 -07:00
Evan Tschannen
1c005d5878
Merge pull request #1584 from alexmiller-apple/spilled-only-peek 
Save TLog resources by letting peek request only spilled data.
 2019-06-20 18:22:31 -07:00
Evan Tschannen
e0be631414 shard the txs tag so that more transaction logs are involved in its recovery 2019-06-19 18:15:09 -07:00
Alex Miller
ce24db3c53 Fully consume parallelPeekMore results before switching back. 2019-06-19 01:30:49 -07:00
Alex Miller
51fd42a4d2 Merge remote-tracking branch 'upstream/master' into spilled-only-peek 2019-06-18 17:33:52 -07:00
mpilman
8576665a90 Revert "Revert "Make protocol version a type"" 
This reverts commit 455bf3b3ec9d5a347b68bf4fa89bf042f5ac312e.
 2019-06-18 14:49:04 -07:00
Alex Miller
455bf3b3ec Revert "Make protocol version a type" 2019-06-18 10:59:17 -07:00
mpilman
da53a92bec Make protocol version a type 
This fixes #1214

The basic idea is that ProtocolVersion is now its own type. This
alone is an improvement as it makes many things more typesafe. For
each version, we can now add breaking features (for example Fearless).
After that, there's no need to test against actual (confusing) version
numbers. Instead a developer can simply test
`protocolVersion->hasFearless()` and this will return true iff the
protocolVersion is newer than the newest version that didn't support
fearless.
 2019-06-16 09:59:15 -07:00
Alex Miller
658e61b394 And now use spilledOnly as a hint to do parallel peeks. 
If there's some spilled data, there's probably a lot of spilled data,
and now we can pull all of it faster.
 2019-05-14 21:03:44 -10:00
Alex Miller
4eb4c03ce5 Save TLog resources by letting peek request only spilled data. 
If a peek is entirely fulfilled from spilled data, then it's likely that
the next peek will be also.  It is thus wasteful for each of these peeks
to call peekMessagesFromMemory, which memcpy's excessively, and then
throw all that data away without using it.

Now, TLogs will give a hint back to peek cursors about if the provided
reply was served entirely from the spilled data, which peek curors then
feed back as the hint into their next request.

At some point, a cursor will send a request for only spilled data, get
an incomplete response, and then be told to send its next request as one
that peeks from memory as well, and then it will fully catch up.
 2019-05-14 15:38:48 -10:00
A.J. Beamon
5f55f3f613 Replace g_random and g_nondeterministic_random with functions deterministicRandom() and nondeterministicRandom() that return thread_local random number generators. Delete g_debug_random and trace_random. Allow only deterministicRandom() to be seeded, and require it to be seeded from each thread on which it is used. 2019-05-10 14:01:52 -07:00
Andrew Noyes
6207d724f8 Fix all -Wunused-variable warnings 2019-04-15 18:13:00 -07:00
Evan Tschannen
b6008558d3 renamed BinaryWriter.toStringRef() to .toValue(), because the function now returns a Standalone<StringRef>() 
eliminated an unnecessary copy from the proxy commit path
eliminated an unnecessary copy from buffered peek cursor
 2019-03-28 11:52:50 -07:00
Evan Tschannen
a2108047aa removed LocalitySetRef and IRepPolicyRef typedefs, because for clarity the Ref suffix is reserved for arena allocated objects instead of reference counted objects. 2019-03-13 13:14:39 -07:00
mpilman
3a0f9839b9 Fix minor IDE build errors 2019-02-19 15:16:59 -08:00
Evan Tschannen
4b5d0b4e2c Merge branch 'release-6.0' 
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/AsyncFileBlobStore.actor.cpp
#	fdbclient/AsyncFileBlobStore.actor.h
#	fdbclient/BlobStore.actor.cpp
#	fdbclient/BlobStore.h
#	fdbclient/HTTP.actor.cpp
#	fdbclient/ManagementAPI.actor.cpp
#	fdbclient/NativeAPI.actor.cpp
#	fdbrpc/LoadBalance.actor.h
#	fdbrpc/batcher.actor.h
#	fdbrpc/fdbrpc.vcxproj
#	fdbrpc/sim2.actor.cpp
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/DataDistributionTracker.actor.cpp
#	fdbserver/SimulatedCluster.actor.cpp
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/masterserver.actor.cpp
 2018-11-10 13:04:24 -08:00
Evan Tschannen
c1bd279a4e addressed review comments 2018-11-04 20:26:23 -08:00
Evan Tschannen
2d9a670774 fix: nested multCursors would improperly hang on getMore, because an inner pop of cursors would not be detected by the outer instance 2018-11-02 13:04:09 -07:00
Robert Escriva
268093a96d Adjust all includes to be relative to the root. 
Remove the use of relative paths.  A header at foo/bar.h could be included by
files under foo/ with "bar.h", but would be included everywhere else as
"foo/bar.h".  Adjust so that every include references such a header with the
latter form.

Signed-off-by: Robert Escriva <rescriva@dropbox.com>
 2018-10-19 17:35:33 +00:00
Evan Tschannen
3922e477a5 Merge branch 'release-6.0' 
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/ManagementAPI.actor.cpp
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/LogSystemDiskQueueAdapter.actor.cpp
#	fdbserver/SimulatedCluster.actor.cpp
#	fdbserver/TLogServer.actor.cpp
 2018-10-03 16:57:18 -07:00
Evan Tschannen
28545e0f8d multi cursors start a get more for the first 10 cursors to hide latency 2018-10-03 13:57:45 -07:00
Alex Miller
535b5701e5 Rewrite all Void _ = wait(...) -> wait(...). 
This takes advantage of the new actorcompiler functionality to avoid
having duplicate definitions of `Void _` when trying to feed the
un-actorompiled source through clang.
 2018-08-14 15:50:26 -07:00
Evan Tschannen
10ae883a68 changed the location of a yield 2018-07-12 17:59:12 -07:00
Evan Tschannen
4fedd05506 added more yields to avoid slow tasks 2018-07-12 17:47:35 -07:00
Evan Tschannen
d47aae27f3 added a yield to getMore() 2018-07-12 16:27:27 -07:00
Evan Tschannen
cd63c7a7cc added a buffered cursor, which efficiently merges lots of peek cursors 2018-07-12 12:09:48 -07:00
Evan Tschannen
c148c865e3 optimized log peek cursors to use much less CPU when using the policy engine 2018-07-11 15:43:55 -07:00
Evan Tschannen
2ec8744ab3 fix: parallel get more needs to verify the begin version matches the end of the previous request, because when a peek cursor expires we lose all history, so the same sequence number could start at different versions 2018-06-25 11:15:49 -07:00
Evan Tschannen
68ac3bdc4c log routers now calculate a precise version to pop for their log router tag 2018-06-21 15:29:46 -07:00
A.J. Beamon
e5488419cc Attempt to normalize trace events: 
* Detail names now all start with an uppercase character and contain no underscores. Ideally these should be head-first camel case, though that was harder to check.
* Type names have the same rules, except they allow one underscore (to support a usage pattern Context_Type). The first character after the underscore is also uppercase.
* Use seconds instead of milliseconds in details.

Added a check when events are logged in simulation that logs a message to stderr if the first two rules above aren't followed.

This probably doesn't address every instance of the above problems, but all of the events I was able to hit in simulation pass the check.
 2018-06-08 11:11:08 -07:00
Evan Tschannen
abcfb0604a fix: cloneNoMore needs to pass useBestSet 2018-04-26 18:32:12 -07:00
Evan Tschannen
7e434348ce fix: storage servers did not properly pull data when configuring from a fearless setup to a non-fearless setup 2018-04-25 18:20:28 -07:00