apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-06-02 19:25:52 +08:00
Code Issues Packages Projects Releases Wiki Activity
9,423 Commits 35 Branches 318 Tags
Commit Graph

9423 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Alex Miller
1547cbcef3 Finish TLog Spill-by-reference design doc, and incorporate feedback. 2020-04-05 23:43:23 -07:00
Alex Miller
0469750087 Merge remote-tracking branch 'upstream/master' into tlog-design-doc 2020-04-05 22:29:59 -07:00
Jingyu Zhou
fe7c5d6bec
Merge pull request #2886 from xumengpanda/mengxu/fr-write-during-read-PR 
Performant restore[25/xx]: Add new tests for parallel restore
 2020-04-02 12:03:03 -07:00
Meng Xu
da7249ed1c FastRestore:Minor revision based on review comments 2020-04-02 11:15:22 -07:00
Meng Xu
38d7f16de0 FastRestore:Add new tests to ctests 2020-04-01 21:34:26 -07:00
Meng Xu
16898408ae FastRestore:Rename tests for new backup by adding NewBackup keyword 2020-04-01 21:31:54 -07:00
Meng Xu
6bce67ca75 FastRestore:Apply clang-format 2020-04-01 21:27:54 -07:00
Meng Xu
1eea388278 Add unit test for splitMutation test file 
Comment out debug trace as well.
 2020-04-01 21:27:18 -07:00
Meng Xu
4cf796e838
Merge pull request #2891 from jzhou77/backup-cmd 
Add an option for fdbbackup to use new backup system
 2020-04-01 16:12:10 -07:00
Meng Xu
4226c713fb MutationRef:typeString:Add MAX_ATOMIC_OP as padding 2020-04-01 10:49:51 -07:00
Meng Xu
ff3fd5c380 FastRestore:Fix:Type can be invalid for typeString array 2020-04-01 10:47:06 -07:00
Meng Xu
7847d70e3a StringRef:toString:Handle empty StringRef 2020-03-31 22:14:05 -07:00
Meng Xu
c69c959428 FastRestore:Fix:It is legal for a backup key not exist in DB 2020-03-31 22:02:17 -07:00
Meng Xu
8a42661761 Disable debugMutation 2020-03-31 19:38:30 -07:00
Jingyu Zhou
78602b0855
Merge pull request #2893 from AlvinMooreSr/binding_pkg 
Binding Test Package fix
 2020-03-31 18:59:06 -07:00
Meng Xu
321b1b6ee7 Add RunRestoreWorkerWorkload into Parallel Restore WriteDuringReadAtomicRestore tests 2020-03-31 18:48:29 -07:00
Alvin Moore
cbcd0abdc0 Fixed the creation of binding test package 2020-03-31 18:40:25 -07:00
Meng Xu
a1d2779f04 Add ParallelRestoreWriteDuringReadAtomicRestore.txt 
Test for fast restore with new backup.
 2020-03-31 18:25:17 -07:00
Meng Xu
0fa3bfa6f3 Increase timeout threshold for ParallelRestoreOldBackupWriteDuringReadAtomicRestore.txt 2020-03-31 18:21:36 -07:00
Meng Xu
25e96a13d3 FastRestore:Fix clearrange on a key mistakenly clear other keys 2020-03-31 17:45:19 -07:00
Meng Xu
212dadc2a1 Fix bug in add mutation on applier 
For clear range mutation, we may clear the right boundary key which should not be cleared.
 2020-03-31 16:51:08 -07:00
Meng Xu
33c4be9c42 Improve debug message for debug mutations 2020-03-31 16:00:51 -07:00
Jingyu Zhou
906174e3e8 Add an option for fdbbackup to use new backup system 
I.e., "-p", or "--partitioned_log" to enable it. By default, old backup system
is used.
 2020-03-31 14:54:32 -07:00
Meng Xu
e286f316b9 Increase generated key length for splitMutation unit test 2020-03-31 14:27:07 -07:00
Meng Xu
b7da76223c Fix a tricky splitMutation bug 
splitMutation result may include the end key of a clearrange mutation
 2020-03-31 13:38:58 -07:00
Meng Xu
ccbbdc4ba4 Unit test:Verify splitMutation by comparing with intersectingRanges result 2020-03-31 12:13:02 -07:00
Xin Dong
8e489e7031
Merge pull request #2870 from dongxinEric/feature/2399/provide-comprehensive-info-about-exclude-in-fdbcli 
Instead of providing partial information about exclude results in dif…
 2020-03-31 11:58:41 -07:00
Xin Dong
18356ec4e6
Apply suggestions from code review 
Co-Authored-By: John Brownlee <mail@johnbrownlee.com>
 2020-03-31 11:11:24 -07:00
A.J. Beamon
09a5629114
Merge pull request #2883 from tclinken/add-getversion-command 
Added fdbcli getversion command
 2020-03-31 07:56:15 -07:00
Meng Xu
faab8a0d19 Add ParallelRestoreOldBackupApiCorrectnessAtomicRestore.txt 
Add tests/ParallelRestoreApiCorrectnessAtomicRestore.txt back which
was mistakenly deleted in the previous commit.
 2020-03-30 21:59:47 -07:00
Meng Xu
73345a7ebd Add new tests for parallel restore 
Add two tests for parallel restore with old backup data
ParallelRestoreApiCorrectnessAtomicRestore.txt
ParallelRestoreOldBackupWriteDuringReadAtomicRestore.txt
 2020-03-30 21:53:26 -07:00
Meng Xu
42df1e7792
Merge pull request #2879 from jzhou77/backup-progress 
Update mutation bytes written for new backups
 2020-03-30 21:42:45 -07:00
Meng Xu
a85652375c
Merge pull request #2872 from jzhou77/backup-fix 
Switch off old mutation logging on proxies for new backups
 2020-03-30 21:42:10 -07:00
Meng Xu
60f6edc3b5
Merge pull request #2860 from zjuLcg/report-conflicting-key-roll-forward 
Report conflicting key roll forward
 2020-03-30 17:33:56 -07:00
tclinken
bee6acb717 Added fdbcli getversion command 2020-03-30 17:12:52 -07:00
Jingyu Zhou
3c76722504 Fix valgrind error of invalid memory access 2020-03-30 16:38:22 -07:00
Jingyu Zhou
3c32835cce Fix decoder for unfinished version batch in a log 
A mutation log's version batch data can be split into multiple blocks, and some
of the blocks can be spread across two mutation logs. Thus, the decoder needs
copy unfinished version batch data from previous file progress to the next one.
 2020-03-30 11:34:51 -07:00
Alvin Moore
f92ef730a4
Merge pull request #2876 from mpilman/bugfixes/cmake-env-set 
Introduce `set_env` function
 2020-03-30 02:13:53 -07:00
Jingyu Zhou
411b4c28ac Update mutation bytes written for new backups 
This will make the log bytes written available to backup status and describe
backup calls.
 2020-03-29 21:23:34 -07:00
Jingyu Zhou
65e3b9192e Add an assert for probably dead code 2020-03-28 21:19:47 -07:00
Jingyu Zhou
241f9c123e Try to find continuous log ranges that cover snapshot begin version 
Backup container can have mutation log files that are not continuous overall,
but contain a continuous range that cover the snapshots. So when determine the
continuous log ranges, try to find one that cover the first snapshot's begin
version.
 2020-03-28 21:19:47 -07:00
Jingyu Zhou
280bc94738 Do not recruit backup workers with wrong tags 
In a rare scenario, the master can recruit backup workers with more tags than
the number of log router tags for an epoch. This can be caused by an
unsuccessful recovery, which uses more tags than the next epoch. When
recruiting for the next epoch, if no progress has been made yet, the recruiting
logic will look back at the previous epoch. If previous epoch has saved past
this epoch's begin version, current epoch's progress is updated with that
information and can result in more tags being inserted to this epoch's
recruitment.
 2020-03-28 21:19:41 -07:00
Jingyu Zhou
3261eae8c1 Initialize contiguousLogEnd to the lowest possible version 
If set to the first file's end version, then we may erroneously set to an
unrestorable version.
 2020-03-28 17:38:02 -07:00
Jingyu Zhou
34745d71fa Fix continuous end version 
When scan from a known version, stop if logs are not continuous. However, if
scan from 0, we should reset minLogBegin to the next file and continue scan
from that file's begin version.
 2020-03-28 17:38:02 -07:00
Jingyu Zhou
abcb458a44 Fix contiguousLogEnd calculation 
The contiguous version should scan from the max of scanBegin version and the
minLogBegin version. Once we found a version that's larger, set it as the
contiguousLogEnd version.
 2020-03-28 17:38:02 -07:00
Jingyu Zhou
efbd01894d Really fix the stuck of backup 2020-03-28 17:38:02 -07:00
Jingyu Zhou
f0d0c863e6 Attempt to fix 2020-03-28 17:38:02 -07:00
Jingyu Zhou
78f74ed6fe Skip copying inside BackupLogsDispatchTask 2020-03-28 17:38:02 -07:00
Jingyu Zhou
a2b8be5dc6 Switch off old mutation logging on proxies for new backups 
Specifically, stop writing LogRange mutation for new backup jobs.
 2020-03-28 17:38:02 -07:00
Jingyu Zhou
cf7404a771
Merge pull request #2871 from xumengpanda/mengxu/parallel-range-log-file-loading-PR 
Performant restore [24/xx]: Process range and log files in parallel for old backup format
 2020-03-28 17:33:50 -07:00