Description
find_package was used to find and link `zlib` library needed to enable
boost::gzip compression filter. However, the code adds dynamic linkage
of zlib shared object with generated binaries (fdbserver for instance).
For now disable the ZLIB find code to effectively disable GZIP compression
support.
Testing
fdbclient/PaxosConfigTransaction.actor.cpp:221:77: runtime error: shift exponent 32 is too large for 32-bit type 'int'
I confirmed that 1 << 30 is not UB
* Introduce "default encryption domain"
Description
In current FDB native encryption data at-rest implementation,
an entity getting encrypted (mutation, KV and/or file) is categorized
into one of following encryption domains:
1. Tenant domain, where, Encryption domain == Tenant boundaries
2. FDB system keyspace - FDB metadata encryption domain
3. FDB Encryption Header domain - used to generate digest for
plaintext EncryptionHeader.
The scheme doesn't support encryption if an entity can't be categorized
into any of above mentioned encryption domains, for instance, non-tenant
mutations are NOT supported.
Patch extend the encryption support for mutations for which corresponding
Tenant information can't be obtained (Key length shorter than TenantPrefix)
and/or mutations do not belong to any valid Tenant
(FDB management cluster data) by mapping such mutations to a
"default encryption domain".
TODO
CommitProxy driven TLog encryption implementation requires every transaction
mutation to contain 1 KV, not crossing Tenant-boundaries. Only exception to
this rule is ClearRange mutations. For now ClearRange mutations are mapped
to 'default encryption domain', in subsequent patch appropriate handling
for ClearRange mutations shall be proposed.
Testing
devRunCorrectness - 100k
Contains the following fixes:
* When handling the special case rollforward where nodes can be rolled
forward even if a majority are at version 0, we don't want to reset
the live version of the node being rolled forward. This is because a
quorum of nodes at version 0 can continue handing out and incrementing
their live version, and if they are rolled forward there is the
potential for them to go back in time in regard to their live version.
So in this one special case, they should maintain their existing live
version.
* Fixes some unseed issues due to fields not being initialized properly.
* Temporarily disables a coordinator restart in the recovery path (in
the coordinated state) due to it causing a timeout. This needs more
investigation in the future.
Specifying the `--no-config-db` option when changing coordinators
through fdbcli will prevent the command from hanging when the
configuration database is not active. Failing to specify this option
when the configuration database is not active will not affect the
correctness of the command, but it will hang instead of returning.
Configuration database data lives on the coordinators. When a change
coordinators command is issued, the data must be sent to the new
coordinators to keep the database consistent.
* ApiTester: test with invalid cluster files
* More asserts in monitorProxies
* ApiTester: Test tampering the cluster file
* Fix update of connection string from the cluster file to use the new connection string only if it valid
* ApiTester: add linker dependency on std++fs
* upgrade_test: no-cleanup-on-error option
* ApiTester: use atomic operations to change and access the transaction handle
Adding the following metrics:
* BlobCipherKeyCache hit/miss
* EKP: KMS requests latencies
* For each component that using encryption, they now need to pass a UsageType enum to the encryption helper methods (GetEncryptCipherKeys/GetLatestEncryptCipherKey/encrypt/decrypt) and those methods will help to log get cipher key latency samples and encryption/decryption cpu times accordingly.
* Assert that arena's appear last in serializer calls
* Fix all occurrences of Arena's not appearing last in serializer call
* Work around issue from Standalone inheriting from Arena privately
* Attempt to fix windows build
Use fb_ prefix instead of detail namespace to scope implementation
details in headers
* Remove API 720 guards for tenants (experimental feature) and the cluster ID special keys (no need to guard)
* Enable the relaxed special key access in transactions that need to use special key-space APIs introduced in 7.2
* ApiTester: enable access to database instances in workloads
* ApiTester: Inject database create errors
* Fix getClusterProtocol and database shared object initialization in case of database create errors
* Check deferred error in all Thread-Safe API calls returning a future
* ApiTester: disable injection of database create errors for tests with old versions
