apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-05-15 02:18:39 +08:00
Code Issues Packages Projects Releases Wiki Activity
21,254 Commits 35 Branches 318 Tags
Commit Graph

87 Commits

Author SHA1 Message Date
A.J. Beamon
70bc67ce34 Update tenant assignment to check data cluster availability. Cleanup errors when performing an invalid operation on a non-metacluster and remove redundant function. 2022-07-18 16:15:33 -07:00
A.J. Beamon
cb499fa5db Merge branch 'main' into feature-metacluster 2022-07-13 15:28:34 -07:00
He Liu
bc5bfaffda
Shard based move (#6981) 
* Shard based move.

* Clean up.

* Clear results on retry in getInitialDataDistribution.

* Remove assertion on SHARD_ENCODE_LOCATION_METADATA for compatibility.

* Resolved comments.

Co-authored-by: He Liu <heliu@apple.com>
 2022-07-07 20:49:16 -07:00
A.J. Beamon
e1a93988ef Merge branch 'main' into feature-metacluster 2022-06-28 14:58:07 -07:00
A.J. Beamon
4b69723a2f Cleanup metadata associated with a cluster when force removing it. Move some metacluster metadata into the normal key-space. Cleanup some todos. 2022-06-24 14:55:57 -07:00
A.J. Beamon
96fad987eb Add new command to get total metacluster capacity. Fix bug where removing a data cluster could make it impossible to create new tenants on it. Add knobs for max data clusters and max tenants. 2022-06-17 10:42:03 -07:00
Xiaoge Su
e493f1c3cd fixup! Add a retry mechanism in changeQuorumChecker and changeQuorum 
This is to fix an issue when recovery and change coordinator key happens
together. The issue will occur when:

1. Recovery starts
2. Coordinator key change transaction started
3. During the recovery the coordinator key is read from cluster file and
   stored in the storage server
4. The cluster controller received `ChangeCoordiatorsRequest`, and
   updated the cluster name with the new value.

at this stage, the value related to coordinator key in storage server and
the worker is inconsistent.

5. changeQuorumChecker is called, which will verify such consistency.
   Since they are different, the call is returning failure and the
   caller, which could be a TEST_CASE, fails.

This is a rare race issue, and it is also noticed that when the
recovery/coordinator key change process is done, the database is in a
proper state which allows changeQuorumChecker behave properly. In this
case, a retry mechanism should be sufficiently fix corresponding test
failures.
 2022-06-14 10:43:13 -07:00
Zhanwei Wang
da346d718f Fix compatibility issue of s3 backup 
1. For v4 signature, http request path should be encoded, currently only parameters are encoded.
2. For v2 signature, http request date header should be in form of either the http date header or x-amz-date, currently in form of ISO 8601
3. Empty response header value cause http_bad_response error.
4. Skip verifying MD5 for range get request and if http response code 206 is returned.
 2022-05-25 09:48:54 -05:00
Josh Slocum
674e6a8fdc Merge branch 'main' into min_shard_bytes_main 2022-04-29 16:00:27 -05:00
A.J. Beamon
79063d5fc4
Better accounting for tenant prefixes when working with key size limits (#6973) 
* Consider tenant prefix sizes when doing key size limit checks

* Fix formatting
 2022-04-27 14:38:59 -07:00
Josh Slocum
ca68fefca2 Decreasing MIN_SHARD_BYTES knob 2022-04-20 18:46:11 -05:00
Dan Lambright
5bdc525353
Merge branch 'main' into vv 2022-04-08 13:16:04 -04:00
Dan Lambright
60c55e0785 Merge remote-tracking branch 'origin/version-vector-prototype' into vv 2022-04-05 11:17:39 -04:00
Renxuan Wang
465ff712b6
Move Hostname to its own files. (#6759) 
* Change DNS cache to use std::map.

Revert commit 90c259d84e95dd35e01149c0a86bd18e82e33930, because if we use unordered_map, toString() can be inconsistent.

* Move ClientKnob::COORDINATOR_HOSTNAME_RESOLVE_DELAY to FlowKnob::HOSTNAME_RESOLVE_DELAY.

* Move Hostname to its own files.

Also, add resolve-related variables and functions in Hostname.
 2022-04-04 19:04:51 -07:00
Josh Slocum
268caa5ac8 fixing shard size knobs outside of simulation 2022-04-04 11:38:18 -05:00
Jingyu Zhou
4fd414a8ed Merge branch 'main' into vv 
Fix Conflicts:
	fdbclient/NativeAPI.actor.cpp
 2022-03-31 09:38:36 -07:00
Jingyu Zhou
cfcf0f152c Merge branch 'main-4a085fc84' into vv 
Fix Conflicts:
	fdbclient/NativeAPI.actor.cpp
	fdbserver/ClusterRecovery.actor.cpp
	fdbserver/MasterInterface.h
	fdbserver/masterserver.actor.cpp
	flow/error_definitions.h
 2022-03-30 22:28:06 -07:00
Jingyu Zhou
b34f4052cd Merge branch 'main-f28dfc12b' into vv 
Fix Conflicts:
	fdbclient/MultiVersionTransaction.actor.cpp
	fdbclient/NativeAPI.actor.cpp
	fdbclient/NativeAPI.actor.h
	fdbserver/storageserver.actor.cpp
 2022-03-30 21:01:25 -07:00
Jingyu Zhou
00b57d4cce Merge branch 'main-67eba5ec7' into vv 
Fix Conflicts:
	fdbclient/CommitProxyInterface.h
	fdbclient/NativeAPI.actor.cpp
	fdbclient/StorageServerInterface.h
	fdbserver/CommitProxyServer.actor.cpp
	fdbserver/storageserver.actor.cpp
 2022-03-30 20:05:55 -07:00
Jingyu Zhou
e9659b5dd4 Merge branch 'master-PR-6500' into vv 
Fix Conflicts:
	fdbclient/CommitProxyInterface.h
	fdbclient/NativeAPI.actor.cpp
	fdbserver/masterserver.actor.cpp
 2022-03-30 14:53:49 -07:00
Evan Tschannen
90983990df
Merge pull request #6680 from sfc-gh-jslocum/bg_client_bindings_improvements 
granule parallelism, metrics, and other improvements to bg client bindings
 2022-03-25 13:44:43 -07:00
Renxuan Wang
2a59c5fd4e
Workers should monitor coordinators in submitCandidacy(). (#6655) 
* Workers should monitor coordinators in submitCandidacy().

* Change re-resolve delay to a knob.
 2022-03-24 19:20:42 -07:00
Josh Slocum
b92c6ea92c granule parallelism, metrics, and other improvements to bg client bindings 2022-03-24 13:44:09 -05:00
Josh Slocum
f27475e2f4 Merge branch 'main' into blob_integration 2022-03-22 11:41:58 -05:00
sfc-gh-tclinkenbeard
a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
Josh Slocum
37e7c80f26 Merge branch 'main' into blob_integration 2022-03-17 18:45:42 -05:00
A.J. Beamon
1e1098ca9a Don't check whether tenants are enabled when watch is called. Support unknown_tenant error. 2022-03-15 09:23:30 -07:00
A.J. Beamon
ecccfd0868 Add cache invalidation to tenant cache. Send tenant ID along with tenant name in requests to validate that the tenant hasn't changed. Fix a few bugs. 2022-03-15 09:23:30 -07:00
A.J. Beamon
2a21126028 Don't apply read prefixes on the client. Cache tenant data locally. 2022-03-15 09:23:30 -07:00
Sreenath Bodagala
bb1a8bd7bd - Set default value of SEND_ENTIRE_VERSION_VECTOR to false. 2022-03-09 19:21:29 +00:00
Josh Slocum
e71b3533f9 Merge branch 'main' into blob_integration 2022-03-09 08:59:56 -06:00
Sreenath Bodagala
91b75ae014 - Intoduce a new knob "CLIENT_KNOBS::SEND_ENTIRE_VERSION_VECTOR" 
- Send the entire version vector on the read version code path
(sequencer -> grv proxy and grv proxy -> client)
 2022-03-08 21:21:10 +00:00
Josh Slocum
665e182bcc TSS Metrics improvements 2022-03-07 11:20:24 -06:00
A.J. Beamon
cdebda35ab
Merge pull request #5725 from sfc-gh-jfu/jfu-grv-cache 
Add transaction option for clients to use cached read versions
 2022-03-04 09:17:27 -08:00
Zhe Wu
9c99c59b0c Fix get key location overloading proxies 2022-02-24 16:35:30 -08:00
Jon Fu
9ab21fe873 remove unused knob 2022-02-23 13:49:53 -05:00
Jon Fu
31d56b2f50 adjust knob setting, revert trace changes, and disable cache if onError retries transactions 2022-02-22 12:42:53 -05:00
Josh Slocum
38a75a8b89 Merge branch 'main' into blob_integration 2022-02-17 17:47:38 -06:00
Jon Fu
d399daebed Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache 2022-02-15 15:09:40 -05:00
Vaidas Gasiunas
092b5cee4b MVC2.0: Rollback added code 2022-02-14 13:50:42 -08:00
Jon Fu
7492b755d8 Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache 2022-02-14 14:06:49 -05:00
Lukas Joswiak
cdc1549282 Fix client timeout errors 2022-02-09 13:43:33 -08:00
Lukas Joswiak
d5a562e6b8 Fix dynamic knobs correctness issues 2022-02-09 13:43:32 -08:00
Bala Namasivayam
9fe7a99e11 Make AWS V4 header default 2022-02-07 17:53:05 -08:00
Bala Namasivayam
a420098fc5 Support AWS v4 header for s3 backup and restore 2022-02-07 17:53:05 -08:00
Jon Fu
ec2bbf0343 clean up some more trace lines and leftover code snippets 2022-02-07 14:50:04 -05:00
Josh Slocum
88cab7fb67 More change feed fetching improvements and optimizations 2022-02-07 11:19:51 -06:00
Jon Fu
5d9843ec32 Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache 2022-01-24 13:20:51 -05:00
Josh Slocum
cf45354833 switched buggified and expected shard size for simulation 2022-01-20 20:37:03 -08:00
Josh Slocum
4bfef29e4c Changed small shards in simulation logic 2022-01-20 20:37:03 -08:00