apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-05-17 03:12:21 +08:00
Code Issues Packages Projects Releases Wiki Activity
21,726 Commits 35 Branches 318 Tags
Commit Graph

257 Commits

Author SHA1 Message Date
Yi Wu
364644673f
Support TLog encryption in commit proxy (#6942) 
This PR add support for TLog encryption through commit proxy. The encryption is done on per-mutation basis. As CP writes mutations to TLog, it inserts encryption header alongside encrypted mutations. Storage server (and other consumers of TLog such as storage cache and backup worker) decrypts the mutations as they peek TLog.
 2022-06-29 14:21:05 -07:00
sfc-gh-tclinkenbeard
1e17f32469 Move StorageMetrics unit tests into cpp file 2022-05-10 13:21:23 -07:00
Renxuan Wang
0a332ee1c1 Add proxy option to backup and restore params. 2022-03-28 17:10:49 -07:00
sfc-gh-tclinkenbeard
a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
sfc-gh-tclinkenbeard
8dcac2f76d Fix typos 2022-03-13 10:02:11 -03:00
A.J. Beamon
250a88e682 Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement. 2022-02-24 12:25:52 -08:00
A.J. Beamon
59503a397e Fix duplicate trace field. 2022-01-05 11:30:21 -08:00
Xiaoge Su
abf73047ca Enforce std:: specifier rather than using namespace 2021-09-16 19:40:28 -07:00
Steve Atherton
3922fa8cdd Fix duplicate TraceEvent detail name. 2021-08-10 17:41:31 -07:00
Steve Atherton
eb5c8eb78d Added Label field for keys and ranges in MutationTracking. 2021-08-10 17:32:32 -07:00
Steve Atherton
faa4154a56 MutationTracking now uses a vector of keys to track. Removed "Mutation" detail from DEBUG_MUTATION() events because they are duplicates of the fields already logged in the returned MutationTracking event, which are now renamed and combined into "Mutation". Removed more toString() calls in TraceEvent detail values. 2021-08-09 23:30:45 -07:00
FDB Formatster
df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Andrew Noyes
79cec09255 Apply clang-tidy's performance-inefficient-vector-operation fix 
I ran this command in my build directory after compiling with
OPEN_FOR_IDE. It took a few small tweaks to get it to compile, which is
outside the scope of this commit.

    $ python run-clang-tidy.py -j $(nproc) -checks='-*,performance-inefficient-vector-operation' -fix
 2021-03-04 03:58:25 +00:00
Andrew Noyes
0ef44739ea Fix OPEN_FOR_IDE build in preparation for using clang-tidy 2021-01-26 02:04:11 +00:00
Richard Chen
c77d9e4abe merge conflicts 2020-12-02 21:53:19 +00:00
sfc-gh-tclinkenbeard
4669f837fa Add uses of makeReference 2020-11-07 22:10:18 -08:00
Richard Chen
41843f07e6 add simulator support for different process versions and ProtocolVersion test 2020-10-12 18:19:31 +00:00
Meng Xu
a3c861b59d
Merge pull request #3777 from apple/release-6.3 
Merge Release 6.3 into master
 2020-09-14 10:32:50 -07:00
Meng Xu
2febbe74ce FastRestoreApplier:Fix conflict range inverted due to invalid memory access 
Rerpot error on loader and applier if not error_code_operation_cancelled error ever happens
 2020-09-09 14:40:21 -07:00
Meng Xu
e69ee87f33 Refactor blob failure to simulateBlobFailure function 2020-09-09 10:54:21 -07:00
Meng Xu
5880bd9fa7 BackupContainer:Fix getSnapshotFileKeyRange_impl on blob http error 2020-09-09 08:47:43 -07:00
Meng Xu
83d1350d8d FastRestore:Handle retriable blob error 2020-09-09 07:29:17 -07:00
Meng Xu
6d58064964 Buggify _parsePartitionedLogFileOnLoader with blob http error 2020-09-09 06:46:08 -07:00
Meng Xu
f3986d201d Buggify backup decodeRangeFileBlock with blob connection error 2020-09-09 06:40:41 -07:00
Evan Tschannen
12edadd059 Merge branch 'release-6.3' 
# Conflicts:
#	CMakeLists.txt
#	fdbclient/Knobs.cpp
#	fdbclient/MasterProxyInterface.h
#	fdbrpc/simulator.h
#	fdbserver/MasterProxyServer.actor.cpp
#	tests/fast/CycleAndLock.txt
#	tests/fast/TxnStateStoreCycleTest.txt
#	tests/fast/VersionStamp.txt
#	tests/slow/ParallelRestoreOldBackupApiCorrectnessAtomicRestore.txt
#	tests/slow/ParallelRestoreOldBackupCorrectnessCycle.txt
#	versions.target
 2020-08-31 19:33:34 -07:00
Meng Xu
7a29a3157f FastRestore:Fix various bugs discovered by enhanced simulation 
1. sendMutation request can be dispatched when the version batch has finished and its data has been deleted;
2. Request scheduler on loader may get into infinite loop because FASTRESTORE_SCHED_LOAD_REQ_BATCHSIZE knob can be set to 0 in simulation
 2020-08-26 15:53:18 -07:00
Meng Xu
f1bd2a18ed Resovle review comments: No functional change 2020-08-26 10:30:31 -07:00
Meng Xu
bd7c07436b FastRestore:Add batchIndex to RestoreAsset for better performance tracking 2020-08-25 09:34:18 -07:00
Meng Xu
778daf20c0 FastRestore:Fix incorrect assert 2020-08-24 19:59:56 -07:00
Meng Xu
996ba2374c FastREstore:Fix:Incorrect condition in printing out FastRestoreLoaderSendMutationToApplierDoneTooLate 2020-08-24 17:27:40 -07:00
Meng Xu
e94261efe5 FastRestore:LoaderScheduler:Add validation on too old requests 2020-08-24 16:32:16 -07:00
Meng Xu
6e3e36c8fc FastRestore:RequestScheduler:Minor code style improvement 2020-08-24 10:45:46 -07:00
Meng Xu
d9ea14ea6c FastRestore:fix:loader can receive reply from vb that has been processed and deleted 2020-08-19 10:39:49 -07:00
Meng Xu
88b94a3e59 FastRestore:Fix:Delayed request can be skipped from processing 
When a request of sending mutations to appliers is delayed and the request for future version batches have been processed,
loader will increase its counter. The previous implementation will use the counter to decide if a request is too old and
have been processed. If the counter is larger than the batch index of the request, the request will be ignored although
it may not have been processed.

This causes correctness test failure for fast restore.
 2020-08-18 23:48:06 -07:00
Meng Xu
e1d150662f FastRestore:small fix on compile error 2020-08-18 16:15:11 -07:00
Meng Xu
a32cf7d293 FastRestore:Loader:Add sanity check before remove batch data 2020-08-18 15:44:22 -07:00
Meng Xu
9b2f667bbe FastRestore:Fix uninitialized variable 2020-08-18 11:58:57 -07:00
Meng Xu
4a0315483b FastRestore:Safeguard when request of earlier vb may be sent after the vb has finished 2020-08-17 22:20:54 -07:00
Meng Xu
7b7490efe7 FastRestore:Debug trace for seg fault 2020-08-17 20:34:33 -07:00
Meng Xu
01255b7ead FastRestoreLoader:Sched:Add trace field and misc code style improvement 2020-08-16 09:43:04 -07:00
Meng Xu
be75fd4894 Workaround flow limitation:state variable must be initialized at the beginning of actors 2020-08-16 08:45:13 -07:00
Meng Xu
c2dfc5a00a Fix compile error 2020-08-16 08:13:28 -07:00
Meng Xu
cb0e44ff42 FastRestoreLoader:Sched send mutations per loadingParam requests 2020-08-16 07:38:51 -07:00
Meng Xu
98167a6d20 FastRestoreLoader:Fix:sched may stuck due to race condition 2020-08-16 06:29:09 -07:00
Meng Xu
015397be74 FastRestore:LoaderScheduler:Control number of load request to release at once 2020-08-15 22:18:48 -07:00
Meng Xu
1e571a5a1a FastRestore:Loader:Kick off scheduler when loader starts to have new requests 2020-08-15 21:57:00 -07:00
Meng Xu
a035e7a872 FastRestore:LoaderScheduler:Add knobs 2020-08-15 21:44:51 -07:00
Meng Xu
7e302b5910 FastRestore:Loader:Add scheduler to dispatch requests based on resource utilization and request priority 2020-08-15 21:21:51 -07:00
Evan Tschannen
2f52c5f79b Merge branch 'release-6.3' 
# Conflicts:
#	fdbserver/RestoreLoader.actor.cpp
 2020-08-11 22:45:43 -07:00
Meng Xu
c27542d4e9 FR:SampleBackup:Send reply when receive duplicate msg 2020-08-07 16:50:36 -07:00