apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-05-22 06:40:01 +08:00
Code Issues Packages Projects Releases Wiki Activity
12,980 Commits 35 Branches 318 Tags
82b6daa16b
Commit Graph

188 Commits

Author SHA1 Message Date
sfc-gh-tclinkenbeard
82b6daa16b First draft of Azure blob storage backup container 2020-10-24 16:47:51 -07:00
sfc-gh-tclinkenbeard
a9607bdcec Explicitly seal classes that inherit but aren't inherited from 2020-10-07 21:58:24 -07:00
sfc-gh-tclinkenbeard
a3de8c333b Use override where applicable in fdbclient 2020-10-07 16:57:38 -07:00
Meng Xu
1fb11858e6 Fix failure caused by merge 2020-09-27 11:03:16 -07:00
Meng Xu
bbc7ce581e Resolve conflicts merging from 6.3 to master 2020-09-25 16:08:31 -07:00
Meng Xu
862336de8f Merge branch 'master' into mengxu/merge-to-master-PR 2020-09-24 17:06:00 -07:00
Young Liu
d4e3e98957 Use new getRestoreSet in RestoreController 2020-09-17 17:56:36 -07:00
Young Liu
71309a79f9 Merge branch 'release-6.3' into range 2020-09-17 17:54:33 -07:00
Meng Xu
8224e17a08 Integrate extended getRestoreSet API into fast restore 
The extended getRestoreSet provides a much smaller set of backup files for small keyrange restore;
This commit integrate it into fast restore so that fast restore does not have to filter out unneeded backup files.
 2020-09-15 09:38:43 -07:00
Meng Xu
f9a5b727a6 Add comments questions and TODOs 2020-09-14 16:28:01 -07:00
Meng Xu
a3c861b59d
Merge pull request from apple/release-6.3 
Merge Release 6.3 into master
 2020-09-14 10:32:50 -07:00
Meng Xu
5880bd9fa7 BackupContainer:Fix getSnapshotFileKeyRange_impl on blob http error 2020-09-09 08:47:43 -07:00
Young Liu
3728ed03dd Resolve comments 2020-09-05 18:55:09 -07:00
Young Liu
1ad5e17458 add support for comparing original and current impls 2020-09-05 11:14:59 -07:00
Jon Fu
b9baa996e3 Merge branch 'master' of https://github.com/apple/foundationdb into jfu-incremental-backup-only 2020-09-01 14:38:02 -04:00
Evan Tschannen
12edadd059 Merge branch 'release-6.3' 
# Conflicts:
#	CMakeLists.txt
#	fdbclient/Knobs.cpp
#	fdbclient/MasterProxyInterface.h
#	fdbrpc/simulator.h
#	fdbserver/MasterProxyServer.actor.cpp
#	tests/fast/CycleAndLock.txt
#	tests/fast/TxnStateStoreCycleTest.txt
#	tests/fast/VersionStamp.txt
#	tests/slow/ParallelRestoreOldBackupApiCorrectnessAtomicRestore.txt
#	tests/slow/ParallelRestoreOldBackupCorrectnessCycle.txt
#	versions.target
 2020-08-31 19:33:34 -07:00
Young Liu
b6c0299d09 Add help message in backup CLI for added options 2020-08-31 09:31:57 -07:00
Young Liu
33aa10b461 Minor optimizations 2020-08-29 20:10:45 -07:00
Young Liu
fd7198d874 Extend backup container interface to support query restorable files set by key ranges 2020-08-29 19:58:07 -07:00
Jon Fu
00c77ba2b4 Added beginVersion cmd line option and addressed code review comments 2020-08-28 14:29:22 -04:00
Meng Xu
ca9b1f5b34 Merge branch 'release-6.3' into mengxu/fr-sched-PR 
Resolve conflict at BackupContainer.actor.cpp
 2020-08-27 16:54:00 -07:00
Meng Xu
369000a125 BackupContainer:Remove link to filename with random string 2020-08-26 15:55:36 -07:00
Meng Xu
a2ab709a0c BackupContainer:Use processId as the process filename 
instead of using a randomly generated string which change every time
when a file is open.

Having too many files will trigger TOO_MANY_FILES error
 2020-08-26 15:54:34 -07:00
Meng Xu
6256bedf8d BackupContainer:Use processId as the process filename 
instead of using a randomly generated string which change every time
when a file is open.

Having too many files will trigger TOO_MANY_FILES error
 2020-08-25 12:25:09 -07:00
Jon Fu
ae999aa118 Merge branch 'master' of https://github.com/apple/foundationdb into jfu-incremental-backup-only 2020-08-19 16:36:47 -04:00
Jon Fu
7dce3a9187 fixed issue with mutations not applying and allow backup to non-empty db 2020-08-11 15:39:21 -04:00
Jon Fu
21635f8a28 update backup restore for local testing 2020-08-04 15:48:43 -04:00
Evan Tschannen
a49cb41de7 Merge branch 'release-6.3' 
# Conflicts:
#	CMakeLists.txt
#	cmake/ConfigureCompiler.cmake
#	fdbserver/Knobs.cpp
#	fdbserver/StorageCache.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	flow/ThreadHelper.actor.h
#	flow/serialize.h
#	tests/CMakeLists.txt
 2020-07-29 00:31:55 -07:00
Andrew Noyes
d2cf700bd4 Fix compiler warnings 2020-07-28 18:30:26 +00:00
Xin Dong
2ac7df8a18
Merge pull request from xumengpanda/mengxu/fr-filesize-PR 
FastRestore:Add trace for file size and bc progress
 2020-07-25 20:08:57 -07:00
Meng Xu
99d8399f4e FastRestore:Add trace for file size and bc progress 2020-07-25 19:12:25 -07:00
Evan Tschannen
e1dedff7b3 Merge branch 'release-6.2' into release-6.3 
# Conflicts:
#	CMakeLists.txt
#	bindings/c/test/mako/mako.c
#	cmake/ConfigureCompiler.cmake
#	documentation/sphinx/source/downloads.rst
#	fdbcli/fdbcli.actor.cpp
#	fdbclient/FileBackupAgent.actor.cpp
#	packaging/msi/FDBInstaller.wxs
#	versions.target
 2020-07-24 12:10:44 -07:00
Oleg Samarin
18964fae2a Unclear error message in fdbbackup if the backup url uses a symlink 2020-07-14 15:39:31 +03:00
Andrew Noyes
6446b4c082 WIP 2020-07-09 22:02:43 +00:00
David Youngworth
f22a8845e4 Finish putting actors in Platform.actor.cpp 2020-05-20 18:14:29 -07:00
David Youngworth
5559643403 Fix rebase issue 2020-05-06 13:39:18 -07:00
David Youngworth
a09f30e48e Make backupContainer's listFiles step asynchronous 2020-05-06 10:42:40 -07:00
Meng Xu
07a9a05683 FastRestore:Agent:Fix restore requests 2020-04-30 16:20:20 -07:00
Meng Xu
6ee78aa3a4 Fix:Disable sanity check backup metadata file 
Which can increase the false positive rate of TooManyFiles error
 2020-04-30 16:16:14 -07:00
Jingyu Zhou
a8becb9027 Use calculated block size for range files in unit tests 2020-04-28 21:13:18 -07:00
Jingyu Zhou
ba261eda36 Fix a backup container unit test 
Write a valid range file instead of random data so that checking its content is
fine.
 2020-04-28 15:39:24 -07:00
Meng Xu
59217ddf1e Remove sanity check on metadata 
The sanity check parses each range file to get the key range of each
range file.
The parsing incurs restore_unsupported_file_version error.

We need to include this sanity check before 6.3 release.
 2020-04-23 14:59:58 -07:00
Jingyu Zhou
9bfc5bbea8 Check RestorableFileSet's key ranges in simulation 
Ranges written in the manifest file should match with actual file content.
 2020-04-21 12:55:40 -07:00
Jingyu Zhou
0938e45c6a Set continuous version to invalidVersion when snapshot version is the target version 2020-04-20 22:26:42 -07:00
Jingyu Zhou
930b175c4c Add range files' key ranges to RestorableFileSet 
Also add continuous logs' begin and end version in RestorableFileSet.
 2020-04-20 22:26:42 -07:00
Jingyu Zhou
a2b867c6f9 Fix a unit test failure 2020-04-20 22:26:42 -07:00
Jingyu Zhou
3063611355 Write range files' begin & end keys to manifest file 
This information can be very useful in knowing the content in these files,
especially for restores.
 2020-04-20 22:26:42 -07:00
Jingyu Zhou
4c66c8c377 Fix backup progress calculation 
The oldest epoch the master gets can assume its begin version is 1, which can
be wrong. In this case, we use the saved backup progress to "true-up" the real
begin version.
 2020-04-20 11:06:46 -07:00
Jingyu Zhou
db2cef844b Write mutation log type as a backup property 
This can solve the problem when listing log files returns empty results.
 2020-04-09 09:29:24 -07:00
Jingyu Zhou
4d06e837dc Remove getPartitionedRestoreSet() API 
Use getRestoreSet() instead for both old and new partitioned logs.
 2020-04-08 20:12:09 -07:00