9972 Commits

Author SHA1 Message Date
A.J. Beamon
e5bf838c83 Fix metacluster register command hints, fix bug in cluster registration on retries, extend the length of the tenant management concurrency test 2022-07-28 21:02:12 -07:00
A.J. Beamon
e8e4f3ad3a Merge branch 'main' into feature-metacluster
# Conflicts:
#	fdbclient/include/fdbclient/Tenant.h
2022-07-28 16:53:29 -07:00
A.J. Beamon
5db1135ac6 Add MetaclusterOperationContext to make it easier to run multi-transaction operations 2022-07-28 16:44:20 -07:00
A.J. Beamon
cd19f2cdc2 Add metacluster awareness of the tenant count limit 2022-07-28 15:03:38 -07:00
He Liu
35a4cb91d5
Disable ShardedRocksDB in simulation when shard_encode_location_metadata is disabled (#7726)
* Disabled tests for ShardedRocks.

Cleaned up ShardedRocks TraceEvent.

Added assertion in ShardManager::validate().

* Added test trace.

* Make sure TraceEvent contains `ShardedRocks`.

* Exclude ShardedRocksDB when SHARD_ENCODE_LOCATION_METADATA is disabled.

Co-authored-by: He Liu <heliu@apple.com>
2022-07-28 13:54:29 -07:00
A.J. Beamon
620467a91b Fix some formatting issues 2022-07-28 13:42:34 -07:00
A.J. Beamon
7f53b340f5 Merge branch 'main' into feature-metacluster
# Conflicts:
#	bindings/python/tests/fdbcli_tests.py
#	fdbcli/TenantCommands.actor.cpp
#	fdbclient/ClientKnobs.cpp
#	fdbclient/include/fdbclient/ClientKnobs.h
#	fdbclient/include/fdbclient/Tenant.h
#	fdbserver/BlobManager.actor.cpp
#	fdbserver/BlobWorker.actor.cpp
#	fdbserver/TenantCache.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/TenantManagementWorkload.actor.cpp
#	flow/include/flow/error_definitions.h
2022-07-28 11:43:21 -07:00
Junhyun Shim
83e94781b4
Merge pull request #6842 from sfc-gh-mpilman/features/authz
Authorization
2022-07-28 19:50:24 +02:00
Xiaoxi Wang
4091a567ab
Merge pull request #7725 from sfc-gh-xwang/fix/main/status
fix storage_metadata disappear because other attribution is missed
2022-07-27 17:14:08 -07:00
Junhyun Shim
631cca6560 More code review suggestions 2022-07-28 01:04:15 +02:00
Xiaoxi Wang
bcf022575c fix storage_metadata disappear because other attribution is missed 2022-07-27 15:55:38 -07:00
Junhyun Shim
1e83d7e1ab Apply code review suggestions 2022-07-28 00:47:14 +02:00
Junhyun Shim
144bb4e110
Apply suggestions from code review
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-07-28 00:45:31 +02:00
Junhyun Shim
c6342a6e5b
Merge branch 'main' into features/authz 2022-07-27 20:51:32 +02:00
Junhyun Shim
27e2bf60c6 Handle case where exception is thrown before tenant prefix is prepended 2022-07-27 20:13:25 +02:00
Jon Fu
af7a644037 Merge branch 'main' of github.com:apple/foundationdb into jfu-tenant-rename-special-keys 2022-07-27 09:37:38 -07:00
A.J. Beamon
7c6b3fb0b8 Merge branch 'main' into feature-metacluster 2022-07-27 08:55:10 -07:00
A.J. Beamon
d39c0b773a Add a limit to the number of tenants that can be created in a cluster 2022-07-27 08:21:03 -07:00
A.J. Beamon
dec6dbfbfb
Merge pull request #7549 from sfc-gh-ajbeamon/feature-tenant-groups
Add support for tenant groups
2022-07-27 07:56:27 -07:00
Josh Slocum
cf0fadab85
Merge pull request #7655 from sfc-gh-jslocum/improved_cf_testing
Improved change feed testing
2022-07-27 08:04:37 -05:00
He Liu
edbc373815
Get ShardedRocks ready for simulation test. (#7679)
* Disabled tests for ShardedRocks.

Cleaned up ShardedRocks TraceEvent.

Added assertion in ShardManager::validate().

* Added test trace.

* Make sure TraceEvent contains `ShardedRocks`.

Co-authored-by: He Liu <heliu@apple.com>
2022-07-26 21:49:33 -07:00
Josh Slocum
1ea43aa878 Merge branch 'main' into improved_cf_testing 2022-07-26 18:12:58 -05:00
Josh Slocum
77956dc7ae
Merge pull request #7639 from sfc-gh-jslocum/cf_metadata_rewrite
Change Feed Metadata Rewrite and adding targeted fault injection
2022-07-26 18:10:37 -05:00
Jon Fu
f0687237fb update comments 2022-07-26 15:41:40 -07:00
Junhyun Shim
5169616b16 Fix unresolved merge conflicts 2022-07-27 00:38:16 +02:00
Junhyun Shim
e2a3fedfc7
Merge branch 'main' into features/authz 2022-07-27 00:08:57 +02:00
Jon Fu
a3b56ebadc reject overlap in transaction case 2022-07-26 15:04:18 -07:00
Josh Slocum
8b5b7a9b92
Merge pull request #7689 from sfc-gh-jslocum/purge_fixes
Granule Purge Fixes
2022-07-26 17:02:24 -05:00
Josh Slocum
5c2681f0b5 formatting 2022-07-26 16:47:07 -05:00
Josh Slocum
0609aa57d1 Merge branch 'main' into improved_cf_testing 2022-07-26 16:44:13 -05:00
Josh Slocum
ba31229797 addressing review comments 2022-07-26 16:37:06 -05:00
Josh Slocum
4d2f90977d
Merge pull request #7656 from sfc-gh-jslocum/cf_bw_operational_fixes
Cf bw operational fixes
2022-07-26 16:24:26 -05:00
Josh Slocum
c32e1da908
Merge pull request #7673 from sfc-gh-jslocum/delta_files_v2
Sorted Delta Files
2022-07-26 16:04:55 -05:00
Jon Fu
6ff90c83b1 remove extra parameters 2022-07-26 13:58:19 -07:00
Jon Fu
6dbaf46ec8 restructure test workload impl and map 2022-07-26 13:53:04 -07:00
Xiaoxi Wang
7125902714
Merge pull request #7709 from sfc-gh-xwang/fix/main/remove-runAfter
remove runAfter actor
2022-07-26 13:38:56 -07:00
Jingyu Zhou
698360daf0
Fix a crash bug during CC shutdown process (#7705)
actor_cancel should be thrown, otherwise the rest code can access invalid
memory.
2022-07-26 12:45:39 -07:00
Josh Slocum
15e7a4b186 addressing review comments 2022-07-26 14:20:35 -05:00
Xiaoxi Wang
7cf6037191 remove runAfter 2022-07-26 12:12:56 -07:00
Jon Fu
0950d82fe5 add documentation and fix merge errors 2022-07-26 11:58:49 -07:00
Jon Fu
21309c2af8 Merge branch 'feature-tenant-groups' of github.com:sfc-gh-ajbeamon/foundationdb into jfu-tenant-rename-special-keys 2022-07-26 11:15:56 -07:00
Xiaoxi Wang
47af598749
Merge pull request #7701 from liquid-helium/resume-data-move-timing
Make sure resumeFromDataMoves() starts after resumeFromShards().
2022-07-26 09:45:14 -07:00
A.J. Beamon
35fd459002 Some test fixes 2022-07-26 09:04:29 -07:00
A.J. Beamon
a64693518a Add support for tenant groups 2022-07-26 09:04:29 -07:00
He Liu
e0b8530ad1 Make sure resumeFromDataMoves() starts after resumeFromShards(). 2022-07-25 22:33:24 -07:00
Chaoguang Lin
5045844ea1
Add -r changeclusterkey tool for snapshot restore (#7687)
* Debug version of fdbserver -r changeclusterkey tool

* Remove debug symbols; move function to Coordinaton.actor.cpp

* Format and add traces

* fix comments
2022-07-25 22:12:28 -07:00
Zhe Wu
410ad5ff18 TLog track unpopped recovery tag 2022-07-25 20:10:18 -07:00
Renxuan Wang
dc9599f2e9
Get network interfaces ready for https proxy. (#7556)
* Move HTTP from fdbclient/ to fdbrpc/.

* Move md5 and libb64 to contrib/.

* Get network interfaces ready for https proxy.

* Rebase
2022-07-25 17:08:32 -07:00
A.J. Beamon
6719e5a85b
Merge pull request #7676 from liquid-helium/ss-shard-management
Fixed compiler warnings.
2022-07-25 13:28:54 -07:00
He Liu
118c2ae806
Fixed memory leak for RocksDb and CheckpointReader. (#7665)
* Fixed memory leak for RocksDb and CheckpointReader.

* Close kvStore at the end of StorageServerCheckpointRestoreTest.
2022-07-25 12:36:29 -07:00