Previously, `status json` would report a machine as excluded when any of
its processes were excluded. But a machine should only be reported as
excluded when all of its processes are excluded.
I tested this change by running a small, three process test cluster. I
excluded one machine, and verified `status json` reported the overall
machine as not excluded.
* Refactor CP::WriteMutation to optimize ACTOR alloc/dealloc
Description
WriteMutation routine is responsible for appending mutations to be persisted
in TLog, the operation isn't a 'blocking' operation, except for few cases
when Encryption is supported by the cluster such
as:
1. Fetch encryption keys to encrypt the mutation.
2. Split ClearRange mutation to respect Encryption domain boundaries.
3. Ensure sanity of already encrypted mutation - simulation limited check.
Patch optimizes the "fast" path by avoiding alloc/dealloc overhead
due to be ACTOR framework support, the penalty happens iff any of above
conditions are met.
Testing
devRunCorrectness - 100K
The actor compiler does not handle this correctly, so disallow it for
now. We could probably support this someday if someone is sufficiently
motivated.
Previously this would miscompile and hopefully cause some kind of syntax
error in the generated code. Now it issues a helpful diagnostic.
Tested by adding an actor with a state variable that tries to use
uniform initialization syntax and observing the error message looks
good. We don't have any infrastructure for testing that the
ActorCompiler rejects something, so I'm not including a test in this
change. We could probably add that relatively easily though.
* Fix a potential memory error
The returned KeyRef should live at least as long as the supplied arena
* Improve keyAfter and singleKeyRange
Avoid duplicating the keyAfter implementation, and share memory between
begin and end for singleKeyRange returning a standalone
* Avoid creating and destroying an arena in a loop
This defeats some of the purposes of Arena's, namely to amortize the
cost of calling malloc and free and to improve cache locality.
* Improve Arena usage
Avoid an arena allocation in keyAfter - instead return a string with
static lifetime. I made sure to return the same memory as was just
brought into cache to inspect whether key == \xff\xff.
Also avoid creating and destroying an arena in a loop for encrypting
idempotency id sets.
* Extend Tlog persistentStorage to persist encryption state
Description
diff-3: Address review comment.
diff-2: Extend ClusterController endpoints to allow query
cluster's encryptionAtRest status
Update Tlog recovery to ensure on-disk encryption
status matches with cluster's cstate persisted
encryptionAtRest
diff-1: Store encryptionAtRestMode state in Coordinators
Major changes proposed are:
1. Extend TLog persistentStorage to persist encryption state
2. Encryption state persisted is derived from corresponding
db-config and relevant SERVER_KNOBS. In near future, knobs
shall be removed.
3. On TLog startup, the persisted encryption state is compared
against cluster configuration, if mismatch, the TLog is killed
and not allowed to rejoin the cluster.
Testing
devRunCorrectness - 100K
DiskQueue: use explicitly sized int32_t instead of int in on-disk format.
It was assumed that the payloadSize field in the page header was 32 bits. This was checked by a static_assert on the size of the PageHeader struct.
This change makes that field size explicit, rather than assuming that sizeof(int) == sizeof(int32_t).
* Upgrade tests: dump thread call stacks of the tester process if it fails to terminate
* ApiTester: log before and after stopping the network thread
* Catch and print exceptions in closeTraceFile; Close trace file at the end of MVC runNetwork
* Change trace event name for MVC runNetwork termination
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
It was assumed that the payloadSize field in the page header was 32 bits.
This was checked by a static_assert on the size of the PageHeader struct.
This change makes that field size explicit, rather than assuming that
sizeof(int) == sizeof(int32_t).
