1
0
mirror of https://github.com/apple/foundationdb.git synced 2025-05-25 08:40:05 +08:00

216 Commits

Author SHA1 Message Date
A.J. Beamon
c7c7bd5f17 Undo change to mark tenant modes non-experimental. Other minor cleanup. 2022-07-19 09:18:53 -07:00
A.J. Beamon
aea4d802c6 Merge branch 'main' into feature-metacluster 2022-06-30 15:08:09 -07:00
Xiaoxi Wang
801c1e8be2 Schema fix 2022-06-29 23:07:04 -07:00
A.J. Beamon
e1a93988ef Merge branch 'main' into feature-metacluster 2022-06-28 14:58:07 -07:00
A.J. Beamon
9f3819752f Change the command to create a metacluster from using 'configure tenant_mode=management' to 'metacluster create <NAME>'. Distribute this name to all processes in a metacluster. Eliminate the tenant mode entirely from metacluster clusters, instead relying on a metacluster registration key. 2022-06-22 12:15:43 -07:00
Xiaoxi Wang
6bb4e341f9
Merge pull request from sfc-gh-xwang/features/ppw-pause-state
Adding paused/running wiggling status to status json and also the last running/paused timestamp
2022-06-16 14:27:18 -07:00
Xiaoxi Wang
a311cc28cc solve some comments 2022-06-16 11:07:21 -07:00
A.J. Beamon
739fc9ce6b Merge branch 'main' into feature-metacluster 2022-06-09 12:39:34 -07:00
Jingyu Zhou
b9ff6bc129 Address AJ's comments 2022-06-08 09:38:32 -07:00
A.J. Beamon
2907aafa60 Merge branch 'main' into feature-metacluster 2022-06-02 12:18:42 -07:00
A.J. Beamon
69261f9f10 Merge branch 'main' into feature-metacluster
# Conflicts:
#	fdbclient/MultiVersionTransaction.actor.cpp
#	fdbclient/MultiVersionTransaction.h
#	fdbserver/tester.actor.cpp
2022-05-19 09:11:30 -07:00
Xiaoxi Wang
382f0fc4a2 merge upstream/main 2022-05-17 10:20:51 -07:00
Sagar Vemuri
ebf11d5d48 Update schemas with the tenants information 2022-05-16 11:09:10 -07:00
A.J. Beamon
85019612d4 Remove subordinate mode and instead use a key in the system key-space to track metacluster membership. Use this key to determine if a cluster is already part of a metacluster or can be configured away from required mode. Disallow configuring to or from a management cluster that has data. 2022-05-11 17:10:55 -07:00
Xiaoxi Wang
2717cee1f9 Merge branch 'features/read-skew' into readaware 2022-05-09 16:12:09 -07:00
Xiaoxi Wang
e6615c29a3 add perpetual wiggler state to json 2022-05-09 11:51:08 -07:00
A.J. Beamon
7e860dc92c Some initial metacluster implementation 2022-05-03 12:53:35 -07:00
Xiaoxi Wang
e9bf7c0851 fix unknown schema check 2022-04-25 17:23:20 -07:00
Xiaoxi Wang
182f244e2e fix schema format 2022-04-25 10:28:45 -07:00
Xiaoxi Wang
e8477e15ce Merge remote-tracking branch 'upstream/main' into readaware 2022-04-25 09:55:52 -07:00
Xiaoxi Wang
adeaf52db3
Merge branch 'apple:main' into ppw-store 2022-04-25 09:40:45 -07:00
Xiaoxi Wang
269e94cb71 add store type to status json 2022-04-24 22:53:06 -07:00
Xiaoxi Wang
0639810b66 merge upstream/main 2022-04-22 11:09:15 -07:00
Bharadwaj V.R
822eb9ec26
Merge branch 'apple:main' into dbcorever 2022-04-22 08:08:34 -07:00
Zhe Wang
6c9ff6ee5e
Add sharded rocksdb type ()
* add-sharded-rocksdb-type

* address comments

Co-authored-by: Zhe Wang <zhewang@Zhes-MacBook-Pro.local>
2022-04-21 22:53:14 -04:00
Bharadwaj V.R
11f2827c82 Make names even across change and add lowest-compatible-version to status json 2022-04-15 13:57:58 -07:00
Bharadwaj V.R
831bee34f8 Add server version to status json 2022-04-13 08:23:02 -07:00
Xiaoxi Wang
aba9d85560 merge main 2022-04-06 09:57:52 -07:00
Steve Atherton
b179813989 Updated status schema and fixed spacing. 2022-04-01 17:21:35 -07:00
He Liu
dd15489605 rename ssd-rocksdb-experimental as ssd-rocksdb-v1. 2022-03-29 10:53:38 -07:00
Xiaoxi Wang
d93b57dd88 conflict solving 2022-03-24 20:45:51 -07:00
Xiaoxi Wang
1b631a9263 solve conflict with main 2022-03-24 16:29:11 -07:00
Josh Slocum
f27475e2f4 Merge branch 'main' into blob_integration 2022-03-22 11:41:58 -05:00
sfc-gh-tclinkenbeard
a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
Josh Slocum
e71b3533f9 Merge branch 'main' into blob_integration 2022-03-09 08:59:56 -06:00
A.J. Beamon
72a34945ce Add the ability to disable tenants. Server processes verify the ID of tenants being read or written. 2022-03-06 21:54:21 -08:00
A.J. Beamon
5fa9d3e1b7 Add a tenant parameter to read and commit requests. Store a map of all tenants on commit proxy and storage servers. Add an option to require tenant mode. 2022-03-06 21:54:21 -08:00
Xiaoxi Wang
e73c0a31e6 add wiggle_server_ids and wiggle_server_addresses in status json 2022-03-02 10:03:23 -08:00
Xiaoxi Wang
f48ba989c3 add status 2022-03-01 15:27:27 -08:00
Josh Slocum
38a75a8b89 Merge branch 'main' into blob_integration 2022-02-17 17:47:38 -06:00
Vaidas Gasiunas
092b5cee4b MVC2.0: Rollback added code 2022-02-14 13:50:42 -08:00
Xiaoxi Wang
6dc5921575
createdTime based storage wiggler ()
* add storagemetadata

* add StorageWiggler;

* fix serverMetadataKey bug

* add metadata tracker in storage tracker

* finish StorageWiggler

* update next storage ID

* change pid to server id

* write metadata when seed SS

* add status json fields

* remove pid based ppw iteration

* fix time expression

* fix tss metadata nonexistence; fix transaction retry when retrieving metadata

* fix checkMetadata bug when store type is wrong

* fix remove storage status json

* format code

* refactor updateNextWigglingStoragePID

* seperate storage metadata tracker and store type tracker

* rename pid

* wiggler stats

* fix completion between waitServerListChange and storageRecruiter

* solve review comments

* rename system key

* fix database lock timeout by adding lock_aware

* format code

* status json

* resolve code format/naming comments

* delete expireNow; change PerpetualStorageWiggleID's value to KeyBackedObjectMap<UID, StorageWiggleValue>

* fix omit start rount

* format code

* status json reset

* solve status json format

* improve status json latency; replace binarywriter/reader to objectwriter/reader; refactor storagewigglerstats transactions

* status timestamp
2022-02-04 15:04:30 -08:00
Ata E Husain Bohra
87ee4cf958 Add new FDB EncryptKeyProxy role
Major changes includes:

1. Add a new FDB role responsible- EncyrptKeyProxy. The role is
   responsible to expose APIs to fetch encyrption keys interacting
   with external Encryption KeyManager interface.
2. The process is a FDB singleton process following similar recruitment
   rules as other singleton processes in the system.
3. Code to recruit the worker process; given the encryption keys are
   needed during recovery (decode TLog records), for now the process
   is co-located in same datacenter as ClusterController.
4. Skeleton process actor code; more functionality will be added in
   subsequent PRs.

NOTE: The code is protected under a SERVER_KNOB with the default
      value as 'false' for now.
2022-01-25 17:38:27 -08:00
Ata E Husain Bohra
936bf5336a
Revert "Revert "Refactor: ClusterController driving cluster-recovery state machine" ()
* Revert "Revert "Refactor: ClusterController driving cluster-recovery state machine""

Major changes includes:
1. Re-revert Sequencer refactor commits listed below (in listed order):
1.a. This reverts commit bb17e194d9c9888e203421290959bd7f2c075d7f.
1.b. This reverts commit d174bb2e06bff01157d16c652073536c54d17f7f.
1.c. This reverts commit 30b05b469c87d9b526b427751c211fb5cf7ff9cd.

2. Update Status.actor to track ClusterController interface to track
   recovery status.
3. Introduce a ServerKnob to define "cluster recovery trace event"
   prefix; for now keeping it as "Master", however, it should allow
   smooth transition to "Cluster" prefix as it seems more appropriate.
2022-01-06 12:15:51 -08:00
Suraj Gupta
640cee2072 Start BM without config change if config is enabled. 2021-12-10 14:00:34 -06:00
Suraj Gupta
fc3376fe8f Move client knob to database config for blob granules. 2021-12-10 14:00:34 -06:00
Steve Atherton
c53f5aa110 Renamed redwood to redwood-1-experimental and file extension to .redwood-v1. 2021-11-16 02:15:22 -08:00
Vaidas Gasiunas
40da5a80f9 Merge remote-tracking branch 'apple/master' into multi-version-client-2 2021-10-26 19:29:10 +02:00
Josh Slocum
5f0ec0612a Merge branch 'feature-range-feed' into blob_full 2021-10-13 15:44:35 -05:00
Vaidas Gasiunas
61fb967484 MVC2.0: Add a clientlib metadata attribute for checksum algorithm 2021-10-07 18:06:22 +02:00