66 Commits

Author SHA1 Message Date
krunal
a21a2dfddd add analytics db to snapshot 2023-11-16 15:01:08 +05:30
krunal
1f8c0d73dd adding tests 2023-08-07 16:23:43 +05:30
krunal1313
c39126ee79 error handling 2023-07-04 17:32:44 +05:30
krunal1313
46a2aa140b moving gzip handling to leader node only 2023-07-04 11:34:10 +05:30
Kishore Nallan
b7f29aeed1 Evented query aggregation skeleton. 2023-05-22 17:52:52 +05:30
Kishore Nallan
6096a96385 Verify status of ext snapshot dir copy. 2023-05-08 22:06:30 +05:30
Kishore Nallan
56184aff26 Allow peers to be reset on error/on-demand. 2023-03-28 16:40:52 +05:30
Kishore Nallan
e3dee78a1b Add resource check to batch indexer as well. 2022-12-10 19:21:54 +05:30
Kishore Nallan
83495c817b Guard for preventing writes when memory/disk thresholds are breached. 2022-11-15 20:25:45 +05:30
Kishore Nallan
9e5c3ed8b2 Status endpoint for tracking lag against leader. 2022-06-30 18:44:59 +05:30
Kishore Nallan
e3d7a5932d Rely on nodes configuration for peer discovery. 2022-06-18 11:03:41 +05:30
Kishore Nallan
ec217bd7d7 Expose snapshot_max_byte_count_per_rpc param. 2022-06-17 15:24:26 +05:30
Kishore Nallan
f3af2319a3 Switch to managed snapshotting.
Ensures that a snapshot does not run when the peers are unhealthy (possibly recovering).
2022-06-17 11:31:48 +05:30
Kishore Nallan
73cc256e2f Add guard for protecting writes from low disk space. 2022-05-10 07:46:39 +05:30
Kishore Nallan
b2acf289ac Move write log persistence logic to batched indexer. 2021-11-03 14:37:38 +05:30
Kishore Nallan
6da39ff51c Expose write stats. 2021-10-02 18:54:30 +05:30
Kishore Nallan
e3bf9f02d6 Expose number of pending writes as metric. 2021-09-21 12:04:08 +05:30
Kishore Nallan
b24df3c26a Allow read/write lags to be updated via config API. 2021-08-11 20:28:47 +05:30
Kishore Nallan
35409f8493 Batched background indexer.
Enables write parallelism across collections.
2021-07-31 21:44:06 +05:30
Kishore Nallan
8e4fc412b1 Retry parsing of nodes config: helps DNS resolution lag. 2021-05-30 20:48:44 +05:30
Kishore Nallan
d6d83ec3ed Resolve hostnames in nodes config to IP automatically. 2021-05-29 20:54:53 +05:30
Kishore Nallan
b3d1a48769 Fix shutdown when node is terminated before becoming leader. 2021-04-26 19:01:32 +05:30
Kishore Nallan
c4deb38189 Handle lag being negative in leader. 2021-04-26 17:19:07 +05:30
Kishore Nallan
2a89acd84e Return 503 for both read and write lag. 2021-04-25 19:15:33 +05:30
Kishore Nallan
aa1cd0acd1 Recover gracefully from a crash during write. 2021-04-23 14:23:00 +05:30
Kishore Nallan
0ae663ab02 Implement rate limiting to protect from write lag. 2021-04-22 07:35:51 +05:30
Kishore Nallan
2591231a0b Fix on-demand snapshot. 2021-04-10 14:24:28 +05:30
Kishore Nallan
5b4b617b59 Refresh catch up status more frequently. 2021-04-10 13:06:56 +05:30
Kishore Nallan
6b793afbae Refactor deferring of batched writes during import. 2021-04-09 16:47:42 +05:30
Kishore Nallan
073667e844 Fix blocking of http thread during writes. 2021-04-08 17:57:02 +05:30
kishorenc
bc0d01bb47 Parameterize thread pool size and collection load configs. 2021-03-29 15:18:47 +05:30
kishorenc
3159712ca3 Remove support for init db snapshot.
Also, handle termination of follower process during import gracefully.
2021-03-26 12:27:04 +05:30
kishorenc
9f159b5d3d Clean shutdown. 2021-03-25 16:05:49 +05:30
kishorenc
f085354824 Copy only state snapshot directory for on-demand snapshots. 2021-03-17 16:12:46 +05:30
kishorenc
0c6a6d483a Protect access to node object. 2021-03-11 18:49:37 +05:30
kishorenc
8d4defa93e Parameterize catchup_min_sequence_diff. 2021-03-04 14:11:28 +05:30
kishorenc
95ee14350a Use shared_ptr to handle req/res life cycles. 2021-02-28 12:48:20 +05:30
kishorenc
e2882c5828 Fix race condition in snapshot restoration. 2021-02-12 19:10:57 +05:30
Jason Bosco
009e67815c Make on-demand snapshot endpoint asynchronous. 2021-01-29 13:58:14 -08:00
Jason Bosco
bcea70ebfd Add windowed stats endpoint 2021-01-22 16:52:27 -08:00
kishorenc
10fd97c32d Node vote API for triggering leader rotation. 2020-12-28 19:20:00 +05:30
kishorenc
8a310bbb82 On demand snapshot API. 2020-12-28 19:19:59 +05:30
kishorenc
efae9055bc Don't mark follower as not caught up for small lags. 2020-09-18 16:52:00 +05:30
kishorenc
a2b421209c Protect API until follower has caught up with leader.
The catch up threshold is configurable as well. Default: 95%
2020-09-17 18:00:32 +05:30
kishorenc
6593d959ec Set min snapshot index gap to 1. 2020-09-10 21:35:55 +05:30
kishorenc
1b01de0c0c Housekeeping tasks. 2020-09-07 21:25:04 +05:30
kishorenc
58b52687ea Handle disconnects during import. 2020-09-04 15:12:09 +05:30
kishorenc
6d7c6b2ea0 Implemented import streaming end-to-end on follower. 2020-09-03 20:44:39 +05:30
kishorenc
02dd4a8a4f Use direct node API for checking leader status.
Otherwise, configuration refresh was getting stuck in a race condition when leader term is not updated while stepping down.
The actual error was "Doing another configuration change".
2020-09-02 11:16:32 +05:30
kishorenc
d9b08d4a90 Handle membership changes more gracefully.
Check for majority going out of sync and reset peers when that happens.
2020-05-10 19:09:40 +05:30