apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-05-14 18:02:31 +08:00
Code Issues Packages Projects Releases Wiki Activity
8,085 Commits 35 Branches 318 Tags
Commit Graph

118 Commits

Author SHA1 Message Date
Jingyu Zhou
11964733b7 WIP: should be divided into smaller commits. 2020-01-22 19:38:45 -08:00
Jingyu Zhou
a797958af6 Update peekLogRouter for backup workers to peek 2020-01-22 19:37:48 -08:00
Jingyu Zhou
a4d6ebe79e Recruit backup worker in newEpoch 2020-01-22 19:37:48 -08:00
Evan Tschannen
afc9713005 Merge branch 'release-6.2' 
# Conflicts:
#	CMakeLists.txt
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/FDBTypes.h
#	fdbserver/LogSystem.h
#	fdbserver/LogSystemPeekCursor.actor.cpp
#	fdbserver/OldTLogServer_6_0.actor.cpp
#	fdbserver/TLogServer.actor.cpp
#	versions.target
 2019-11-06 13:45:37 -08:00
Evan Tschannen
1c873591be fixed a compiler error 2019-11-05 18:32:15 -08:00
Evan Tschannen
a8ca47beff optimized memory allocations by using VectorRef<Tag> instead of std::vector<Tag> 2019-11-05 18:07:30 -08:00
Evan Tschannen
457896b80d remote logs use bufferedCursor when peeking from log routers to improve performance 
bufferedCursor performance has been improved
 2019-11-04 19:47:45 -08:00
Jingyu Zhou
cd3f1e33d4 Refactor deserialization of TagsAndMessages 
Consolidate deserialization of TagsAndMessages in the structure itself and
change both TLog and ServerPeekCursor to use it.
 2019-09-04 14:55:05 -07:00
Jingyu Zhou
4a63de16e9
Merge pull request #1945 from xumengpanda/mengxu/tLog-code-read-v2 
Add comments to DiskQueue and tLog
 2019-08-08 13:24:32 -07:00
Evan Tschannen
4c9a392f05 the master checks the popped version of the txsTag before recovering the txnStateStore, to avoid restoring data that is later found to be popped 2019-08-05 17:01:48 -07:00
Meng Xu
c9c50ceff8 Comments:Add comments to DiskQueue 
No functional change.
 2019-08-01 15:20:01 -07:00
Evan Tschannen
653d9be6e2 we cannot pop old generations because it breaks forced recoveries 2019-07-31 18:27:36 -07:00
Evan Tschannen
1ea3ce8f9c txs pops also go to the old generations of tlogs to reduce the chance we have to restart txnStateStore recovery 2019-07-31 18:06:39 -07:00
Evan Tschannen
7ac7eb82f2 fix: buffered cursor would start multiple bufferedGetMore actors 
advance all of the cursors to the poppedVersion
 2019-07-30 14:42:05 -07:00
Evan Tschannen
9e3ec2cb33 fix: when resetting the peekCursor, we cannot discard the popped data if the adapter has already processed data 2019-07-30 13:25:25 -07:00
Evan Tschannen
1d326e3dc8 removed debugging message 2019-07-30 12:42:50 -07:00
Evan Tschannen
5d79e4141f fix: buffered cursor messageVersion should be set to the version we will be at after exhausting everything in messages 2019-07-30 12:38:44 -07:00
Evan Tschannen
45f7b41b48 fix: multi-cursor could discard popped commits after already returning data 2019-07-29 21:36:42 -07:00
Evan Tschannen
5bb322b483 implement popped on bufferedCursor 2019-07-29 21:19:47 -07:00
Evan Tschannen
28df2c35bb
Merge pull request #1855 from alexmiller-apple/sharded-txs-safe-upgrade 
Make sharded txsTag upgradeable and downgradeable
 2019-07-26 13:29:39 -07:00
sramamoorthy
9afd162e2f remove snap v1 related code 2019-07-25 17:29:31 -07:00
Alex Miller
95487861be Make sharded txsTag gated on TLogVersion::V4. 
To allow a potential 6.2 -> 6.1 rollback.
 2019-07-16 19:09:53 -07:00
Alex Miller
9396eedd11 Const some random functions that are trivially const. 
For code hygiene reasons only.
 2019-07-16 19:09:09 -07:00
Evan Tschannen
15e894c724 Merge in master 2019-07-05 15:49:24 -07:00
Alex Miller
bf883d7055 Merge remote-tracking branch 'upstream/master' into flowlock-api 2019-06-25 14:26:50 -07:00
Alex Miller
7a500cd37f A giant translation of TaskFooPriority -> TaskPriority::Foo 
This is so that APIs that take priorities don't take ints, which are
common and easy to accidentally pass the wrong thing.
 2019-06-25 02:47:35 -07:00
Evan Tschannen
1c005d5878
Merge pull request #1584 from alexmiller-apple/spilled-only-peek 
Save TLog resources by letting peek request only spilled data.
 2019-06-20 18:22:31 -07:00
Alex Miller
26343f557a Update getMore() contract. 
MultiCursor already did this.
 2019-06-20 17:48:24 -07:00
Evan Tschannen
e0be631414 shard the txs tag so that more transaction logs are involved in its recovery 2019-06-19 18:15:09 -07:00
Alex Miller
51fd42a4d2 Merge remote-tracking branch 'upstream/master' into spilled-only-peek 2019-06-18 17:33:52 -07:00
mpilman
8576665a90 Revert "Revert "Make protocol version a type"" 
This reverts commit 455bf3b3ec9d5a347b68bf4fa89bf042f5ac312e.
 2019-06-18 14:49:04 -07:00
Alex Miller
455bf3b3ec Revert "Make protocol version a type" 2019-06-18 10:59:17 -07:00
mpilman
da53a92bec Make protocol version a type 
This fixes #1214

The basic idea is that ProtocolVersion is now its own type. This
alone is an improvement as it makes many things more typesafe. For
each version, we can now add breaking features (for example Fearless).
After that, there's no need to test against actual (confusing) version
numbers. Instead a developer can simply test
`protocolVersion->hasFearless()` and this will return true iff the
protocolVersion is newer than the newest version that didn't support
fearless.
 2019-06-16 09:59:15 -07:00
sramamoorthy
3d5998e9dd tlog: when pops are disabled, store them & replay 
In Tlogs, disable pop is done whlie taking snapshots. Earlier, tlogs
were ignoring the pops if it got pop requests when pops were
disabled. In this change, instead of ignoring the pop - it remembers
the list of pops in-memory and plays them once the popping is
enabled.
 2019-05-28 22:07:46 -07:00
sramamoorthy
4bc4c615da exec op to all tlog, restore change in test &other 
- exec operation to go to all the TLogs
- minor bug fix in tlog
- restore implementation for the simulator
- restore snap UID to be stored in restartInfo.ini
- test cases added
- indentation and trace file fixes
 2019-05-28 22:07:46 -07:00
sramamoorthy
69edefe68b Snapshot based backup and resotre implementation 2019-05-28 22:07:46 -07:00
Jingyu Zhou
b8e7fc1b84 Refactor: add std:: qualifier and use emplace_back 2019-05-17 09:38:50 -10:00
Alex Miller
4eb4c03ce5 Save TLog resources by letting peek request only spilled data. 
If a peek is entirely fulfilled from spilled data, then it's likely that
the next peek will be also.  It is thus wasteful for each of these peeks
to call peekMessagesFromMemory, which memcpy's excessively, and then
throw all that data away without using it.

Now, TLogs will give a hint back to peek cursors about if the provided
reply was served entirely from the spilled data, which peek curors then
feed back as the hint into their next request.

At some point, a cursor will send a request for only spilled data, get
an incomplete response, and then be told to send its next request as one
that peeks from memory as well, and then it will fully catch up.
 2019-05-14 15:38:48 -10:00
Jingyu Zhou
8b5449e608 Fix review comments for PR #1473 2019-04-29 16:45:42 -07:00
Jingyu Zhou
5462f560e7 Add pseudo locality for log routers and tlogs 
This changes the logic of pop operations from log routers (LG):
- LG pops tagLocalityLogRouterMapped from TLogs;
- TLog converts tagLocalityLogRouterMapped back to tagLocalityLogRouter before
  popping.

Later when we add more psuedo localities, the same pattern can be used.
 2019-04-23 21:35:56 -07:00
Jingyu Zhou
010f825aff Remove pseudoLocalities from LogSet, TLogSet, and CoreTLogSet 2019-04-21 10:41:07 -07:00
Jingyu Zhou
7befce6bf1 More pseudoLocalities and refactors. 2019-04-21 10:41:07 -07:00
Jingyu Zhou
966ec30fcc Add pseudoLocalities for special tag consumers 2019-04-21 10:41:07 -07:00
Jingyu Zhou
d19b0cf1c1 Refactor LogSet with two new constructors 2019-04-21 10:41:07 -07:00
Jingyu Zhou
0b1984978a Small code refactoring. 2019-04-21 10:41:07 -07:00
Jingyu Zhou
ec1bc5cfca Add LogSystemType enum 2019-04-21 10:41:07 -07:00
Evan Tschannen
b6008558d3 renamed BinaryWriter.toStringRef() to .toValue(), because the function now returns a Standalone<StringRef>() 
eliminated an unnecessary copy from the proxy commit path
eliminated an unnecessary copy from buffered peek cursor
 2019-03-28 11:52:50 -07:00
Evan Tschannen
5a00f567be fix CheckSatelliteTagLocation 2019-03-20 09:30:11 -07:00
Balachandar Namasivayam
f9560e1abd Addressed Review Comments 2019-03-19 15:23:14 -07:00
Balachandar Namasivayam
5471725db5 Support config where the primary and remote DC's can be used as satellites. 2019-03-18 12:17:59 -07:00