apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-05-15 18:32:18 +08:00
Code Issues Packages Projects Releases Wiki Activity
16,046 Commits 35 Branches 318 Tags
Commit Graph

381 Commits

Author SHA1 Message Date
sfc-gh-tclinkenbeard
382533c4e1 Add FDB_ prefix to BOOLEAN_PARAM macros 2021-07-09 05:42:14 -07:00
sfc-gh-tclinkenbeard
020371a78f Merge remote-tracking branch 'origin/master' into add-boolean-param 2021-07-07 16:50:51 -07:00
Trevor Clinkenbeard
f5ade03538
Merge pull request #4233 from sfc-gh-tclinkenbeard/encrypt-backup-files 
Added AsyncFileEncrypted
 2021-07-07 13:28:28 -07:00
sfc-gh-tclinkenbeard
8b2d8448d7 Move BackupAgentBase method implementations to .cpp file 2021-07-04 19:09:03 -07:00
sfc-gh-tclinkenbeard
8370d92c20 Expand use of BOOLEAN_PARAM in backup code 2021-07-04 18:30:55 -07:00
sfc-gh-tclinkenbeard
cadfdc551e Use BOOLEAN_PARAM for backup agents 2021-07-04 16:27:26 -07:00
sfc-gh-tclinkenbeard
49c680b252 Use BOOLEAN_PARAM for TaskBucket 2021-07-04 13:14:25 -07:00
sfc-gh-tclinkenbeard
8cc40e3a2b Expand use of BOOLEAN_PARAM 2021-07-02 21:41:50 -07:00
sfc-gh-tclinkenbeard
f5aa3df917 Add --encryption_key_file command line argument to fdbbackup and fdbrestore 2021-06-25 15:11:03 -07:00
sfc-gh-tclinkenbeard
53f5cd2453 Support passing encryption file to BackupContainer::openContainer 2021-06-25 14:11:21 -07:00
sfc-gh-tclinkenbeard
594e8944ae Move RestoreWorkerInterface into fdbserver 2021-05-30 11:51:47 -07:00
Jingyu Zhou
bb076115c9 Only enable backup worker when using partitioned logs 
This addresses issue #4849.
 2021-05-21 16:40:29 -07:00
sfc-gh-tclinkenbeard
5c2d7b6080 Create RangeResult type alias 2021-05-03 13:14:16 -07:00
RenxuanW
5e045bd21c Move the read of logsOnly and inconsistentSnapshotOnly to a loop where other RestoreConfig vars are fetched. 2021-04-29 17:00:34 -07:00
RenxuanW
44197644be Reset transaction after a commit and before start a read. 2021-04-29 12:19:58 -07:00
RenxuanW
2f3d70c084 Fix the logic of getting firstConsistentVersion. 
First consistent version should be:

- In a logs-only restore, it is the begin version the user said to start applying logs for;
- In an inconsistent-snapshot-only restore, if all range files have the same version, then it is that version, otherwise unknown (use -1);
- If using both range files and logs, then it is the highest version of any range file in the RestoreSet’s ranges vector.
 2021-04-27 11:27:57 -07:00
RenxuanW
719f810676 Rename incrementalBackupOnly to onlyAppyMutationLogs in all restore configs and functions. 2021-04-26 12:30:46 -07:00
RenxuanW
41ca11c3e5 Implement restoring an inconsistent snapshot as a real feature. 2021-04-22 13:53:37 -07:00
RenxuanW
bc43fa99ac Move commit to its own try loop. 2021-04-21 17:37:58 -07:00
RenxuanW
b90f61d740 Move commit to its own try loop. 2021-04-21 15:50:25 -07:00
RenxuanW
7c4b5b0337 Add first consistent version in restore status. 
First consistent version is the max of versions in RestoreFileSet.
 2021-04-21 14:32:13 -07:00
RenxuanW
ba60f18ebf Merge remote-tracking branch 'upstream/master' into backup-agent 2021-04-20 22:54:47 -07:00
Trevor Clinkenbeard
81fbe9ceaa
Merge pull request #4684 from sfc-gh-satherton/restore-range-fixes 
Restore target range handling bug fixes
 2021-04-20 11:57:13 -07:00
Steve Atherton
75425b5a24
Merge pull request #4620 from RenxuanW/renxuan/first-pr 
Control backup's initial snapshot interval via backup cmd argument.
 2021-04-19 23:59:02 -07:00
Steve Atherton
3f54a4a6dc Throw an error if an empty range set is passed to restore(). 2021-04-19 21:52:38 -07:00
Steve Atherton
c2c9ca4362 Assert was incorrect. Restore ranges must begin with the restore prefix to remove. 2021-04-19 17:01:20 -07:00
RenxuanW
ab4c5ff90e For better readability 2021-04-19 14:06:50 -07:00
RenxuanW
03c031a09d Update getCurrentVersion_impl 
- If the restore is in the running state, then the current version is the getApplyBeginVersion()
- If the restore is in the completed state, the current version is the restore target version which comes from the restoreVersion() property.
- If the restore is in any other state, the current version can be reported as -1 as you have done.
 2021-04-19 13:43:51 -07:00
RenxuanW
0378dc0a50 Report the current version in the restore status. 2021-04-14 22:19:39 -07:00
RenxuanW
97b995fb4f
Update fdbclient/FileBackupAgent.actor.cpp 
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
 2021-04-14 13:49:59 -07:00
RenxuanW
a285d6019e We cannot put 2 Future functions in the same wait if the second one uses the first's result. 
Before this change:
20210414-180825-renxuan-7451fad7aed4f0c7           compressed=True data_size=22960315 duration=732 ended=146 fail=10 fail_fast=10 max_runs=100000 pass=46 priority=100 remaining=0 runtime=0:01:12 sanity=False started=147 stopped=20210414-180937 submitted=20210414-180825 timeout=5400 username=renxuan

After this change:
20210414-192849-renxuan-cbe0f71ad5c48286           compressed=True data_size=22959419 duration=4261266 ended=106778 fail=1 fail_fast=10 max_runs=100000 pass=99999 priority=100 remaining=0 runtime=0:24:49 sanity=False started=106963 stopped=20210414-195338 submitted=20210414-192849 timeout=5400 username=renxuan
 2021-04-14 13:10:56 -07:00
RenxuanW
9737212e51 The default value of the first snapshot interval should be 0 rather than -1. 2021-04-14 10:56:42 -07:00
RenxuanW
a0430536f1 Remove knob BACKUP_INIT_SNAPSHOT_INTERVAL_SEC. 2021-04-14 10:41:41 -07:00
RenxuanW
ebf37594f7 Change initialSnapshotIntervalSeconds from knob to a backup argument. 2021-04-13 19:22:13 -07:00
RenxuanW
c8b27e71c5 Revert TraceEvent 
We've found the problem (issue #4640), so we no longer need the TraceEvent.
 2021-04-12 15:03:47 -07:00
RenxuanW
dc00d99626 Log FileBackupLogRangeStart before calling getLogRanges() in . 
It will tell us if or why this function is legitimately trying to use too much ram. getLogRange() should normally return about 20 items in the result. If the inputs are trash, it could return far more.

If it isn’t the case, then there’s something else wrong that has corrupted something such that when we try to allocate memory.
 2021-04-09 15:02:45 -07:00
RenxuanW
fadc9cccee Use knob RESTORE_IGNORE_LOG_FILES in restore. 
Rename IGNORE_LOG_FILES to RESTORE_IGNORE_LOG_FILES. Also, this knob should be used in regular restore, not parallel restore.
 2021-04-07 10:05:56 -07:00
RenxuanW
1b6ad42db8 Use a knob to completely ignore log files 2021-04-06 19:07:01 -07:00
RenxuanW
edb3dd4414 Control backup's initial snapshot interval via knob. 2021-04-05 18:25:18 -07:00
Jingyu Zhou
2b4744ad8a Reword comments. 2021-03-29 13:43:43 -07:00
Jingyu Zhou
55ef40864f Add comments for touched functions 2021-03-29 10:09:07 -07:00
Jingyu Zhou
1a0ef39e5d Fix keyRangesFilter with logsOnly flag 
Remove the Sev 40 trace event.
 2021-03-27 21:48:27 -07:00
Jingyu Zhou
a9d8806828 Use the restored range in the actual restore 2021-03-25 10:07:25 -07:00
Steve Atherton
720c0b3a4f
Added const. 
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
 2021-03-24 14:36:55 -07:00
Steve Atherton
a6136ffc62 Applied clang-format to recent changes. 2021-03-24 06:51:08 -07:00
Steve Atherton
17a19021fb Added mutation batch filtering during restore, to the extent possible where each restore mutation log task only sees one block of data (usually 1MB) which can contain incomplete and therefore unfilterable mutation sets. 2021-03-24 06:38:50 -07:00
A.J. Beamon
3751ecd11f
Merge pull request #4503 from sfc-gh-ajbeamon/merge-release-6.3-into-master 
Final merge of release-6.3 into master
 2021-03-16 09:29:05 -07:00
A.J. Beamon
25c4880ebe Merge branch 'release-6.3' into merge-release-6.3-into-master (temporarily discard all changes to BackupContainer.actor.cpp) 
# Conflicts:
#	fdbclient/BackupContainer.actor.cpp
#	fdbserver/Knobs.h
 2021-03-15 16:41:04 -07:00
Trevor Clinkenbeard
766b9d24d6
Merge branch 'master' into remove-invalid-backup-status-fields 2021-03-14 18:57:03 -07:00
Markus Pilman
cfbaaa6001 Merge remote-tracking branch 'origin/release-6.2' into release-6.3 2021-03-10 14:30:47 -07:00