147 Commits

Author SHA1 Message Date
He Liu
c3a68d661e
Physical Shard Move (#6264)
Physical Shard Move part I: Checkpoint creation, transfer and restore.
2022-03-15 13:03:23 -07:00
Tao Lin
e2c7c30faf
GetMappedRange support serializable & check RYW & continuation (#6181) 2022-03-10 10:05:44 -08:00
A.J. Beamon
3ae98189af Server processes send unknown_tenant responses back to clients, which is meant to be retried after updating the tenant cache. Fix bug where key selectors could be truncated after applying the tenant prefix. 2022-03-06 21:54:21 -08:00
A.J. Beamon
72a34945ce Add the ability to disable tenants. Server processes verify the ID of tenants being read or written. 2022-03-06 21:54:21 -08:00
A.J. Beamon
c99c7cf55f Apply tenant prefixes on the storage server. The commit proxy can serve tenant location requests. Fix some bugs. 2022-03-06 21:54:21 -08:00
A.J. Beamon
5fa9d3e1b7 Add a tenant parameter to read and commit requests. Store a map of all tenants on commit proxy and storage servers. Add an option to require tenant mode. 2022-03-06 21:54:21 -08:00
Ata E Husain Bohra
936bf5336a
Revert "Revert "Refactor: ClusterController driving cluster-recovery state machine" (#6191)
* Revert "Revert "Refactor: ClusterController driving cluster-recovery state machine""

Major changes includes:
1. Re-revert Sequencer refactor commits listed below (in listed order):
1.a. This reverts commit bb17e194d9c9888e203421290959bd7f2c075d7f.
1.b. This reverts commit d174bb2e06bff01157d16c652073536c54d17f7f.
1.c. This reverts commit 30b05b469c87d9b526b427751c211fb5cf7ff9cd.

2. Update Status.actor to track ClusterController interface to track
   recovery status.
3. Introduce a ServerKnob to define "cluster recovery trace event"
   prefix; for now keeping it as "Master", however, it should allow
   smooth transition to "Cluster" prefix as it seems more appropriate.
2022-01-06 12:15:51 -08:00
Andrew Noyes
28971c5181 Fix memory leak. Closes #4482 2021-12-22 15:04:00 -08:00
Tao Lin
9b0a9c4503
Return error when getRangeAndFlatMap has more & Improve simulation tests (#6029) 2021-12-03 12:50:07 -08:00
Josh Slocum
a82845af43 Merge branch 'master' into bg_bindings 2021-12-01 16:55:28 -06:00
Josh Slocum
7f4fcc8c2c Added FDBResult and made readBlobGranules use it 2021-12-01 16:22:05 -06:00
Steve Atherton
867999a41a Rename wrong_format_version to unsupported_format_version. 2021-11-16 03:25:54 -08:00
Evan Tschannen
964d0209ca
Merge pull request #5637 from sfc-gh-ljoswiak/features/data-loss-prevention
Data loss protection when joining new cluster
2021-11-15 15:26:32 -08:00
Jingyu Zhou
02d0c43bc2
Merge pull request #5982 from sfc-gh-tclinkenbeard/improve-error-descriptions
Make snapshot errors more descriptive
2021-11-15 13:18:19 -08:00
Evan Tschannen
94a51e57a5 Merge branch 'master' into feature-changefeed-empty-versions
# Conflicts:
#	fdbclient/StorageServerInterface.h
2021-11-14 19:13:05 -08:00
Evan Tschannen
6909754b21 changefeeds now have a whenAtLeast function for efficiently learning when the version has updated but no mutations have been committed 2021-11-14 19:08:46 -08:00
sfc-gh-tclinkenbeard
dc756228f2 Make snapshot errors more descriptive 2021-11-14 13:46:17 -08:00
Josh Slocum
329091e14f Merge branch 'master' into bg_bindings 2021-11-11 10:13:37 -06:00
Josh Slocum
b8ac4213a1 Switched BG APIs to transaction instead of database 2021-11-11 08:59:06 -06:00
Tao Lin
fdb3b72e35 Introduce GetRangeAndFlatMap to push computations down to FDB
Re-introduce #5609
2021-11-09 13:52:28 -08:00
Lukas Joswiak
3988b11fd6 Cleanup 2021-11-09 12:29:48 -08:00
Lukas Joswiak
30867750b5 Add protection against storage and tlog data deletion when joining a new cluster 2021-11-09 12:29:47 -08:00
Tao Lin
586cc3b102
Revert "Introduce GetRangeAndFlatMap to push computations down to FDB" 2021-11-04 08:46:56 -07:00
Tao Lin
6c98e35893 Rename Hop to FlatMap 2021-11-03 13:32:01 -07:00
Tao Lin
0853661d13 Introduce getRangeAndHop to push computations down to FDB 2021-11-03 13:21:16 -07:00
Vaidas Gasiunas
40da5a80f9 Merge remote-tracking branch 'apple/master' into multi-version-client-2 2021-10-26 19:29:10 +02:00
Vaidas Gasiunas
37bc41abbb Merge remote-tracking branch 'apple/master' into multi-version-client-2 2021-10-26 18:51:43 +02:00
Josh Slocum
773886515e Merge branch 'feature-range-feed' into blob_full_clean 2021-10-22 11:07:51 -05:00
Vaidas Gasiunas
39b2cb8125 Merge remote-tracking branch 'apple/master' into multi-version-client-2 2021-10-21 17:45:32 +02:00
Evan Tschannen
f1158371a7 Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
# Conflicts:
#	flow/error_definitions.h
2021-10-21 00:55:12 -07:00
Evan Tschannen
3ebabb6edc fixed incorrect use of change feed errors 2021-10-20 22:37:31 -07:00
Evan Tschannen
3f7df58a77 fixed a number of issues 2021-10-19 13:56:52 -07:00
Josh Slocum
5f0ec0612a Merge branch 'feature-range-feed' into blob_full 2021-10-13 15:44:35 -05:00
Evan Tschannen
5c642f706e Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
# Conflicts:
#	fdbcli/fdbcli.actor.cpp
2021-10-09 19:34:16 -07:00
Steve Atherton
45e19c8072 Merge branch 'master' of https://github.com/apple/foundationdb into kvs-write-version
# Conflicts:
#	fdbserver/VersionedBTree.actor.cpp
2021-10-07 20:48:21 -07:00
Vaidas Gasiunas
816e8703d6 MVC2.0: Operation to download a client library from the system keyspace to a file 2021-10-06 18:01:46 +02:00
Vaidas Gasiunas
cda0a5f931 Operation to upload client library binary in to system keyspace 2021-10-06 18:01:46 +02:00
sfc-gh-tclinkenbeard
8e523c5bae Fixed grv_proxy_failed error message 2021-09-12 17:33:25 -07:00
sfc-gh-tclinkenbeard
368e217adf Fixed commit_proxy_failed error message 2021-09-12 17:33:25 -07:00
Josh Slocum
c2d1d1704f Merge branch 'feature-range-feed' into blob_full 2021-09-10 11:21:52 -05:00
Steve Atherton
b06cf7a328 Redwood commit versions are now FDB commit versions for storage instances. Redwood no longer makes internal commits during Pager or BTree recovery so that commit version is always set by the user. 2021-09-10 03:28:46 -07:00
Evan Tschannen
ac5b580e2d Merge branch 'master' into feature-range-feed
# Conflicts:
#	fdbcli/fdbcli.actor.cpp
#	fdbclient/StorageServerInterface.cpp
#	fdbclient/StorageServerInterface.h
#	fdbserver/ApplyMetadataMutation.cpp
#	fdbserver/TLogServer.actor.cpp
#	flow/error_definitions.h
2021-09-09 23:13:22 -07:00
Evan Tschannen
af0792b1f5 fixes that allow change feeds to pass correctness in tests with no failures 2021-09-03 17:24:33 -07:00
Josh Slocum
3b011408f8 Added sequence numbers and locks to blob worker and manager 2021-08-27 16:33:07 -05:00
sfc-gh-tclinkenbeard
79ba9c4e3a Add GetGenerationQuorum to get generation from a quorum of config nodes 2021-08-09 10:04:35 -07:00
sfc-gh-tclinkenbeard
3c1cabf041 Coordinator lets client know if it cannot communicate with cluster controller 2021-07-13 17:26:28 -07:00
sfc-gh-tclinkenbeard
c5b612510d Added invalid_encryption_key_file error 2021-06-26 11:14:23 -07:00
sfc-gh-tclinkenbeard
4105a9f530 Improved error handling test coverage in LocalConfiguration 2021-06-16 13:10:14 -07:00
sfc-gh-tclinkenbeard
03be6ce8da Remove KnobValueRef::Clear class 2021-06-02 21:36:34 -07:00
sfc-gh-tclinkenbeard
a775f92fca Merge remote-tracking branch 'origin/master' into config-db 2021-06-01 15:39:34 -07:00