970 Commits

Author SHA1 Message Date
A.J. Beamon
dec6dbfbfb
Merge pull request #7549 from sfc-gh-ajbeamon/feature-tenant-groups
Add support for tenant groups
2022-07-27 07:56:27 -07:00
Josh Slocum
cf0fadab85
Merge pull request #7655 from sfc-gh-jslocum/improved_cf_testing
Improved change feed testing
2022-07-27 08:04:37 -05:00
He Liu
edbc373815
Get ShardedRocks ready for simulation test. (#7679)
* Disabled tests for ShardedRocks.

Cleaned up ShardedRocks TraceEvent.

Added assertion in ShardManager::validate().

* Added test trace.

* Make sure TraceEvent contains `ShardedRocks`.

Co-authored-by: He Liu <heliu@apple.com>
2022-07-26 21:49:33 -07:00
Josh Slocum
1ea43aa878 Merge branch 'main' into improved_cf_testing 2022-07-26 18:12:58 -05:00
Josh Slocum
77956dc7ae
Merge pull request #7639 from sfc-gh-jslocum/cf_metadata_rewrite
Change Feed Metadata Rewrite and adding targeted fault injection
2022-07-26 18:10:37 -05:00
Josh Slocum
35056060b0 fixing specific unit test 2022-07-26 17:05:46 -05:00
Josh Slocum
0609aa57d1 Merge branch 'main' into improved_cf_testing 2022-07-26 16:44:13 -05:00
Josh Slocum
c32e1da908
Merge pull request #7673 from sfc-gh-jslocum/delta_files_v2
Sorted Delta Files
2022-07-26 16:04:55 -05:00
A.J. Beamon
a64693518a Add support for tenant groups 2022-07-26 09:04:29 -07:00
Yao Xiao
98bff116a4
Disabled unsupported tests. (#7693) 2022-07-25 21:57:47 -07:00
Josh Slocum
095a5a4868 First version of key-sorted delta files 2022-07-22 11:43:49 -05:00
He Liu
7a8be255cd
Ss shard management (#7340)
* Storage server shard management with physical shards.

* Cleanup.

* Resolved comments.

* Added `UnlimintedCommitBytes`.

Co-authored-by: He Liu <heliu@apple.com>
2022-07-22 09:30:44 -07:00
Josh Slocum
3f8f44dd16 Merge branch 'main' into improved_cf_testing 2022-07-20 08:19:52 -05:00
Josh Slocum
fd9201f60b Merge branch 'main' into cf_metadata_rewrite 2022-07-20 07:55:00 -05:00
Dan Lambright
7e296c2367
simple replication (#7625) 2022-07-19 18:30:34 -04:00
Josh Slocum
78d4d85f3b Adding non-tss delay injection to SS as well 2022-07-19 09:59:14 -05:00
Josh Slocum
0d9bb9f4a5 Added targeted storage server restarts at critical metadata points 2022-07-19 08:33:43 -05:00
Trevor Clinkenbeard
1eec6f993d
Merge pull request #7572 from sfc-gh-akejriwal/tenantquota
Introduce storage quotas per tenant
2022-07-18 20:00:22 -07:00
Ata E Husain Bohra
f6f117592d BlobFile Encryption and compression support
- Limit verbose logging under DEBUG_MACRO
 - Update/Add code documentation

Description

Testing
2022-07-14 17:04:14 -07:00
Ata E Husain Bohra
24b2de8de8 BlobFile Encryption and compression support
Description

Testing
2022-07-14 17:04:14 -07:00
Ankita Kejriwal
bb05321d24 Introduce storage quotas per tenant.
This change adds:
* ability to store the mapping from tenants to quota in the system keyspace,
* a setter and getter function
* a new workload to test this functionality

FDBCORE-2437
2022-07-14 16:35:12 -07:00
Yi Wu
7d7ce0909f
Restart tests carry forward encryption knobs value (#7497)
Previously to get around the issue that EKP is not present when restart test switching encryption from on to off and read encrypted data, EKP was made to start in simulation regardless of encryption knob. This PR revert that change, and instead force restart test not to change encryption knob, by passing previous encryption knob through restartInfo.ini file. Also since we don't allow downgrading an encrypted cluster to previous version, disable encryption in downgrade tests.

Also adding an assert to allow reading encrypted mutations only if encryption knob is on. We may reconsider allowing switching encryption on/off for existing cluster, but for now we don't allow it.
2022-07-14 14:45:17 -07:00
Josh Slocum
142ed7713e Initial version of ChangeFeedOperations workload 2022-07-13 11:16:53 -05:00
Josh Slocum
9831be866f Adding RandomMoveKeys blob granule cycle test and moving lower value fast/ tests to rare/ 2022-07-11 15:49:55 -05:00
Dan Lambright
96bd3dcbf0
custom loopback cluster script (#7550) 2022-07-11 13:02:37 -04:00
Vaidas Gasiunas
1e8feb9cb8
Generate a shim library for the FDB C API (#7506)
* Adding sources of the Implib.so project

* Run C unit tests and API tests with the shim library

* Reuse compilation of C test binaries with and without shim library

* Resolve client library path from an environment variable

* Refactoring: Reusable module for downloading FDB binaries

* Testing client shim library with current version and last release version

* Tests for specifying client library over an environment variable

* Enable C shim library tests on ARM

* Restore the original path for including fdb_api.hpp
2022-07-08 16:28:35 +02:00
He Liu
bc5bfaffda
Shard based move (#6981)
* Shard based move.

* Clean up.

* Clear results on retry in getInitialDataDistribution.

* Remove assertion on SHARD_ENCODE_LOCATION_METADATA for compatibility.

* Resolved comments.

Co-authored-by: He Liu <heliu@apple.com>
2022-07-07 20:49:16 -07:00
sfc-gh-tclinkenbeard
72f5335460 Remove TPCC workloads 2022-06-29 22:31:56 -07:00
sfc-gh-tclinkenbeard
086e4bff06 Merge remote-tracking branch 'origin/main' into global-tag-throttling3 2022-06-28 10:18:13 -07:00
He Liu
61d621a4c4
Fix read iterator pool race (#7455) 2022-06-27 14:51:26 -07:00
Vaidas Gasiunas
e28a8401fb
Update coordinator list from cluster file (#7382)
* Log failed connection attempts in monitorProxies

* Update coordinator list from the cluster file after failing to connect to all coordinators

* Wiggle and upgrade test with legacy version monitoring; updating tests to use 7.1.9

* Update coordinator list from the cluster file: addressing review comments

* Update coordinator list from the cluster file: addressing review comments

* Wait on future for all setAndPersistConnectionString calls
2022-06-23 09:22:09 +02:00
sfc-gh-tclinkenbeard
2391e58fb2 Merge remote-tracking branch 'origin/main' into global-tag-throttling3 2022-06-21 10:09:15 -07:00
sfc-gh-tclinkenbeard
a025d7bdbc Ignore GlobalTagThrottling.toml test 2022-06-19 23:10:54 -07:00
Andrew Noyes
83aceb216c
Use absl::GetStackTrace for slow task profiler (#7374)
* Make SlowTask workload runnable in joshua

* Remove SignalSafeUnwind, and use absl::GetStackTrace for slow task profiler
2022-06-15 14:53:52 -07:00
Yao Xiao
ddbecb69ad
Ignore ShardedRocksDBTest. (#7381) 2022-06-13 23:13:11 -07:00
sfc-gh-tclinkenbeard
df71a49bf6 Merge remote-tracking branch 'origin/main' into global-tag-throttling3 2022-06-13 10:03:10 -07:00
sfc-gh-tclinkenbeard
75c858eb2c Differentiate between different quotas in GlobalTagThrottling workload 2022-06-13 10:01:55 -07:00
Andrew Noyes
e0eedc6a37
Fix asan message detection (#7338)
* Fix asan message detection

* Fix heap-buffer-overflow

Call to strlen on a not necessarily null terminated string

* Fix gcc build
2022-06-08 09:35:30 -07:00
Markus Pilman
d141347500
Merge pull request #7282 from Doxense/fix-windows-tests
Fix windows tests
2022-06-08 08:18:47 -06:00
Dan Adkins
bd47f390bd
Add simulation test for three_data_hall configuration (#7305)
* Add simulation test for 1 data hall + 1 machine failure case.

* Disable BUGGIFY for DEGRADED_RESET_INTERVAL.

A simulation test discovered a situation where machines attempting to connect
to a dead coordinator (with a well-known endpoint) were getting themselves
marked degraded. This flapping of the degraded state prevented recovery from
completing, as it started over any time it noticed that tlogs on degraded
hosts could be relocated to non-degraded ones.

bin/fdbserver -r simulation -f tests/rare/CycleWithDeadHall.toml -b on -s 276841956
2022-06-06 13:14:49 -07:00
sfc-gh-tclinkenbeard
2ae5a6c86b Add GlobalTagThrottling test 2022-06-03 13:12:48 -07:00
Mohamed Oulmahdi
3139e28aeb Fix TestRunner.py typos 2022-05-31 13:29:25 +02:00
Josh Slocum
ffa4255c65 Added blob metadata concept as new secret type, and verified blob workers can load it 2022-05-27 15:15:56 -05:00
Josh Slocum
85af0a25b2 Enabling BM to understand tenant boundaries, and changing BlobGranuleCorrectness to use tenants 2022-05-25 17:16:56 -05:00
Josh Slocum
49b50ae8b0 Disabling default tenant in blob granule tests 2022-05-25 17:16:56 -05:00
Yao Xiao
6bf1e3921f
Add skeleton for sharded rocksdb KVS. (#7181) 2022-05-23 15:29:23 -07:00
sfc-gh-tclinkenbeard
8194460915 Update is_unknown_knob in test_argument_parsing.py 2022-05-23 13:41:48 -07:00
Junhyun Shim
02b2f97e99
mTLS test helpers (#7218)
* Add TLS option to (local_cluster|tmp_cluster).py

* Add TLS-enabled C API test
2022-05-23 12:47:51 +02:00
Vaidas Gasiunas
5205b565ab
Upgrade Tests: Testing downgrade from 7.2 to 7.1 (#7179) 2022-05-18 11:59:29 +02:00
Vaidas Gasiunas
9966e3f7f7 Api Tester: print workload progress statistics in regular time intervals 2022-05-17 17:35:01 +02:00