Meng Xu
2febbe74ce
FastRestoreApplier:Fix conflict range inverted due to invalid memory access
...
Rerpot error on loader and applier if not error_code_operation_cancelled error ever happens
2020-09-09 14:40:21 -07:00
Meng Xu
e69ee87f33
Refactor blob failure to simulateBlobFailure function
2020-09-09 10:54:21 -07:00
Meng Xu
5880bd9fa7
BackupContainer:Fix getSnapshotFileKeyRange_impl on blob http error
2020-09-09 08:47:43 -07:00
Meng Xu
83d1350d8d
FastRestore:Handle retriable blob error
2020-09-09 07:29:17 -07:00
Meng Xu
6d58064964
Buggify _parsePartitionedLogFileOnLoader with blob http error
2020-09-09 06:46:08 -07:00
Meng Xu
f3986d201d
Buggify backup decodeRangeFileBlock with blob connection error
2020-09-09 06:40:41 -07:00
Meng Xu
7a29a3157f
FastRestore:Fix various bugs discovered by enhanced simulation
...
1. sendMutation request can be dispatched when the version batch has finished and its data has been deleted;
2. Request scheduler on loader may get into infinite loop because FASTRESTORE_SCHED_LOAD_REQ_BATCHSIZE knob can be set to 0 in simulation
2020-08-26 15:53:18 -07:00
Meng Xu
f1bd2a18ed
Resovle review comments: No functional change
2020-08-26 10:30:31 -07:00
Meng Xu
bd7c07436b
FastRestore:Add batchIndex to RestoreAsset for better performance tracking
2020-08-25 09:34:18 -07:00
Meng Xu
778daf20c0
FastRestore:Fix incorrect assert
2020-08-24 19:59:56 -07:00
Meng Xu
996ba2374c
FastREstore:Fix:Incorrect condition in printing out FastRestoreLoaderSendMutationToApplierDoneTooLate
2020-08-24 17:27:40 -07:00
Meng Xu
e94261efe5
FastRestore:LoaderScheduler:Add validation on too old requests
2020-08-24 16:32:16 -07:00
Meng Xu
6e3e36c8fc
FastRestore:RequestScheduler:Minor code style improvement
2020-08-24 10:45:46 -07:00
Meng Xu
d9ea14ea6c
FastRestore:fix:loader can receive reply from vb that has been processed and deleted
2020-08-19 10:39:49 -07:00
Meng Xu
88b94a3e59
FastRestore:Fix:Delayed request can be skipped from processing
...
When a request of sending mutations to appliers is delayed and the request for future version batches have been processed,
loader will increase its counter. The previous implementation will use the counter to decide if a request is too old and
have been processed. If the counter is larger than the batch index of the request, the request will be ignored although
it may not have been processed.
This causes correctness test failure for fast restore.
2020-08-18 23:48:06 -07:00
Meng Xu
e1d150662f
FastRestore:small fix on compile error
2020-08-18 16:15:11 -07:00
Meng Xu
a32cf7d293
FastRestore:Loader:Add sanity check before remove batch data
2020-08-18 15:44:22 -07:00
Meng Xu
9b2f667bbe
FastRestore:Fix uninitialized variable
2020-08-18 11:58:57 -07:00
Meng Xu
4a0315483b
FastRestore:Safeguard when request of earlier vb may be sent after the vb has finished
2020-08-17 22:20:54 -07:00
Meng Xu
7b7490efe7
FastRestore:Debug trace for seg fault
2020-08-17 20:34:33 -07:00
Meng Xu
01255b7ead
FastRestoreLoader:Sched:Add trace field and misc code style improvement
2020-08-16 09:43:04 -07:00
Meng Xu
be75fd4894
Workaround flow limitation:state variable must be initialized at the beginning of actors
2020-08-16 08:45:13 -07:00
Meng Xu
c2dfc5a00a
Fix compile error
2020-08-16 08:13:28 -07:00
Meng Xu
cb0e44ff42
FastRestoreLoader:Sched send mutations per loadingParam requests
2020-08-16 07:38:51 -07:00
Meng Xu
98167a6d20
FastRestoreLoader:Fix:sched may stuck due to race condition
2020-08-16 06:29:09 -07:00
Meng Xu
015397be74
FastRestore:LoaderScheduler:Control number of load request to release at once
2020-08-15 22:18:48 -07:00
Meng Xu
1e571a5a1a
FastRestore:Loader:Kick off scheduler when loader starts to have new requests
2020-08-15 21:57:00 -07:00
Meng Xu
a035e7a872
FastRestore:LoaderScheduler:Add knobs
2020-08-15 21:44:51 -07:00
Meng Xu
7e302b5910
FastRestore:Loader:Add scheduler to dispatch requests based on resource utilization and request priority
2020-08-15 21:21:51 -07:00
Meng Xu
c27542d4e9
FR:SampleBackup:Send reply when receive duplicate msg
2020-08-07 16:50:36 -07:00
Meng Xu
fccebd9b80
FR:Use ByteSampleInfo to accurately sample backup
2020-08-06 20:45:46 -07:00
Meng Xu
87bde08695
FR:fix:sending samples may have broken promise error
...
The error should not skip the ack reply from loader to controller, saying
the file has been loaded.
Otherwise, restore controller can get stuck at waiting for the ack reply.
2020-08-06 10:41:18 -07:00
Meng Xu
859c3145e2
FR:Fix samples not send reply and false positive failures
2020-08-05 11:32:04 -07:00
Meng Xu
879998d01b
FR:fix compilation error
2020-08-04 23:04:38 -07:00
Meng Xu
7992cef025
FR:Fix sample network pkg can be too big
2020-08-04 22:35:21 -07:00
Meng Xu
fec3de9b7b
Fix RestoreLoader loadedLogBytes counter
2020-07-31 16:00:15 -07:00
Jingyu Zhou
7279beb662
Add old splitMutation for unit test
2020-07-08 17:57:20 -07:00
Jingyu Zhou
d6dd88c2b9
Move building of KeyRangeMap out of the loop
...
So that it is only done once.
2020-07-08 14:16:10 -07:00
Jingyu Zhou
a187eeebee
Add assertions back and comments for splitMutation
2020-07-08 11:08:23 -07:00
Jingyu Zhou
e78cc9ee4b
Use KeyRangeMap for splitMutation
...
This simplifies code by reuse KeyRangeMap class.
2020-07-08 09:53:34 -07:00
Meng Xu
a5febfbc75
Remove spurious comments
2020-06-29 10:26:01 -07:00
Meng Xu
8f5ee1f4aa
Refine trace events for better perf diagnosis
2020-06-29 10:22:07 -07:00
Meng Xu
82dfb5ce3f
FastRestore:Update process metrics for restore master
2020-06-28 12:37:04 -07:00
Meng Xu
bc98c84346
RestoreLoader release data early and revert Lower priority for RestoreApplierReceiveMutations actor
...
A quick evalution shows lowering priority for receive mutation actor does not help restore speed but hurt it.
2020-06-28 11:12:06 -07:00
Meng Xu
026a735703
Test perf without gracefully clear memory
2020-06-27 16:15:56 -07:00
Meng Xu
d6381de16f
RestoreLoader:Clear memory per loading param
2020-06-27 15:51:19 -07:00
Meng Xu
f18a0a242d
FastRestore:Loader frees mutation memory immediately after send mutations
2020-06-27 15:19:08 -07:00
Meng Xu
e57dba00bd
FastRestore:Lower priority for RestoreApplierReceiveMutations actor
2020-06-27 15:16:38 -07:00
Meng Xu
5860a5b4db
FastRestore:Suppress or mute spammy trace events
2020-06-24 22:10:54 -07:00
Meng Xu
c13ba7aeb4
FastRestore:Cosmetic code style improvement
...
Also mute some trace events
2020-06-24 21:25:37 -07:00
