214 Commits

Author SHA1 Message Date
Meng Xu
fbb6e8f39d FastRestore:Create low memory situation in simulation on purpose 2020-02-26 14:54:38 -08:00
Meng Xu
06495b90ae FastRestore:Loader:Use isSchedulable to guard OOM
And trigger delayed actors that are blocked on memory to recheck memory.
2020-02-26 14:35:05 -08:00
Meng Xu
a354f6ffa2 FastRestore:Applier:Use isSchedulable to guard OOM 2020-02-26 14:12:56 -08:00
Meng Xu
ca726fc68e FastRestore:Introduce OOM protection
An actor is schedulable to run if the current worker has enough resourc, i.e.,
the worker's memory usage is below the threshold;
Exception: If the actor is working on the current version batch, we have to schedule
the actor to run to avoid dead-lock.
Future: When we release the actors that are blocked by memory usage, we should release them
in increasing order of their version batch.
2020-02-26 14:09:18 -08:00
Meng Xu
fbf5020af9 FastRestore:Applier:Add fetchKeys counter 2020-02-26 11:37:40 -08:00
Meng Xu
6bd4703a9f FastRestore:Resolve review comments 2020-02-20 14:27:34 -08:00
Meng Xu
d5d26f589f FastRestore:Cosmetic change to improve code readability 2020-02-19 15:43:51 -08:00
Meng Xu
7897b1658f FastRestore:Applier:Rename new apply actor names 2020-02-19 15:29:32 -08:00
Meng Xu
e4258d73f5 FastRestore:Applier:Remove applying actors that do not have good perf 2020-02-19 15:27:59 -08:00
Meng Xu
fe75a4cafb FastRestore:Apply clang-format 2020-02-19 15:22:52 -08:00
Meng Xu
03f699f2f9 Merge branch 'master' into mengxu/fast-restore-applier-multi-applying-PR 2020-02-19 15:22:33 -08:00
Meng Xu
94d799552e FastRestore:Apply clang-format against master 2020-02-18 16:41:59 -08:00
Meng Xu
132f5aa9ba FastRestore:Improve trace name and cosmetic change 2020-02-18 16:41:19 -08:00
Meng Xu
31a6ec34b7 Merge branch 'master' into mengxu/fast-restore-agent-PR 2020-02-18 16:17:59 -08:00
Meng Xu
c603b20e7e FastRestore:Resolve review comments 2020-02-18 14:08:27 -08:00
Meng Xu
c34a69df32 FastRestore:Applier:Remove unused func 2020-02-13 23:06:56 -08:00
Meng Xu
3e2c19630a FastRestore:Applier:atomicOp can work on an empty key 2020-02-13 23:05:54 -08:00
Meng Xu
b57583a504 FastRestore:Applier:Handle multiple gets in parallel 2020-02-13 23:05:31 -08:00
Meng Xu
53f427c319 FastRestore:Applier:fix getAndComputeStagingKeys 2020-02-13 22:11:30 -08:00
Meng Xu
0d668ea0c3 FastRestore:Applier:Add more trace for perf tracking 2020-02-13 15:50:10 -08:00
Meng Xu
0b27786811 FastRestore:Applier:Minor change for clang-format 2020-02-13 13:17:32 -08:00
Meng Xu
b5e60585aa FastRestore:Applier:Fix precompute mutation result 2020-02-13 12:57:47 -08:00
Meng Xu
b1b44d4477 FastRestore:Applier:Handle CompareAndClear atomicOp 2020-02-13 11:11:29 -08:00
Meng Xu
58dad5373b FastRestore:Applier:Handle CompareAndClear atomicOp 2020-02-13 11:06:20 -08:00
Meng Xu
d3c01763d9 FastRestore:Applier:Handle version stamped key values 2020-02-13 10:48:36 -08:00
Meng Xu
b008df97eb FastRestore:Applier:Multiple set-clear mutations at same version 2020-02-13 10:13:46 -08:00
Meng Xu
238b2cb8e4 FastRestore:Applier:Fix various bugs
1. segmentation error
2. there exist mutations that is not set or clear or atomicOp, precompute result should ignore them.
2020-02-13 10:00:23 -08:00
Meng Xu
acf34319c1 FastRestore:Applier:Precompute mutations and apply in parallel
Precompute mutations received by an applier;
Only apply the final result to the destination DB;
Execute multiple txns in parallel to apply final results to the destination DB.
2020-02-12 22:47:48 -08:00
Meng Xu
2bc82ffd70 FastRestore:Applier:Store received mutation by key 2020-02-12 14:12:38 -08:00
Meng Xu
c0f75d77b1 FastRestore:Applier:Intro StagingKey struct 2020-02-12 13:57:18 -08:00
Meng Xu
3e6bbe9e5b FastRestore:Applier:Use real size for atomic op 2020-02-11 15:51:32 -08:00
Meng Xu
cda8fc189e FastRestore:AtomicOp:Intro weighted size for atomicOp
atomicOp has an amplified performance overhead to the cluster,
for example, an ADD operation can be small, but SS has to load
the value to do the operation and the value can be large.
2020-02-11 12:48:05 -08:00
Meng Xu
e76b6d824a FastRestore:Assign priority to actors to prioritize vb work
When we pipeline multiple version batches, we should prevent a later
version batch from blocking the earlier version batch by consuming
CPU resources.

To achive the above, we should assign higher priority to actors
in later phases in a version batch.

Because restore master will not invoke an actor at a later phase unless
the actors at the earlier phases have been finished. This priority assignment
will not cause dead lock.
2020-02-10 20:29:23 -08:00
Meng Xu
325bd52939 FastRestore:Applier:Count appliedTxns 2020-02-10 17:13:20 -08:00
Meng Xu
dbce1e9974 FastRestore:Applier:Add metrics counter and proc counter 2020-02-10 16:38:26 -08:00
Meng Xu
9b7a00a64f FastRestore:Mute trace when apply to db 2020-02-06 20:52:24 -08:00
Meng Xu
dc848f4297 FastRestore:Disable verbose trace for perf. measurement 2020-02-06 20:50:23 -08:00
Meng Xu
cab9d51e06 Merge branch 'master' into mengxu/fast-restore-pipeline-PR 2020-01-27 18:16:26 -08:00
Meng Xu
141609e80a FastRestore:Improve code style and fix typos 2020-01-27 18:13:14 -08:00
Meng Xu
b04e98771e FastRestore:Replace FastRestoreOpConfig with Knobs
And randomize value for the rest of knobs
2020-01-24 14:24:34 -08:00
Meng Xu
4bf579a6d5 FastRestore:Fix race condition in pipeline
Master should not start asking appliers to apply mutations at batchIndex
until all appliers have applied mutations at (batchIndex - 1).
Otherwise, mutations may not be applied in increasing order of versions,
because appliers at different batch index can have overlapped key ranges.
2020-01-23 16:34:45 -08:00
Meng Xu
e011f39829 FastRestore:Add sanity check and trace events 2020-01-23 16:03:41 -08:00
Meng Xu
009fcdeb16 FastRestore:Sanity check each restore asset is processed exactly once 2020-01-21 17:17:45 -08:00
Meng Xu
022783b449 Start batches in reverse order for testings and code cleanup 2020-01-21 14:49:40 -08:00
Meng Xu
4ac92d223b Cleanup batch buffer for each restore request 2020-01-21 14:49:36 -08:00
Meng Xu
1a130b0df3 FastRestore:Fix race condition on handleApplyToDBRequest 2020-01-17 17:01:09 -08:00
Meng Xu
bfbf2164c4 FastRestore:Applier buffer data for multiple batches 2020-01-17 17:01:01 -08:00
Meng Xu
67e913c3d5 Change LoadingParam struct and endVersion definition
1) Remove endVersion field because it has been included in RestoreAsset;

2) Ensure endVersion in VersionBatch and RestoreAsset is always exclusive;

3) Revise ASSERT in laoder and applier in situations when the dummy commit version
is endVersion, to avoid false positive ASSERT failure.
2020-01-07 11:48:03 -08:00
Meng Xu
8d6f511816 FastRestore:Resolve review comment
Filter out range mutations that do not overlap with the restore range.
Small changes on format.
2019-12-22 20:09:10 -08:00
Meng Xu
ddcf3fdd80 FastRestore:Apply clang format 2019-12-20 22:00:36 -08:00