Josh Slocum
b92c6ea92c
granule parallelism, metrics, and other improvements to bg client bindings
2022-03-24 13:44:09 -05:00
Ata E Husain Bohra
017709aec6
Introduce BlobCipher interface and cipher caching interface ( #6391 )
...
* Introduce BlobCipher interface and cipher caching interface
diff-3: Update the code to avoid deriving encryption key periodically.
Implement EncyrptBuf interface to limit memcpys.
Improve both unit test and simulation to better code coverage.
diff-2: Add specific error code for OpenSSL AES call failures
diff-1: Update encryption scheme to AES-256-CTR. Minor
updates to Header to capture more information.
Major changes proposed are:
1. Introduce encyrption header format.
2. Introduce a BlobCipher cipher key representation encoding
following information: baseCipher details, derived encryption cipher
details, creationTime and random salt.
3. Introduce interface to support block cipher encrytion and decrytion
operations. Encyrption populates encryption header allowing client to
persist them on-disk, this header is then read allowing decryption
on reads.
4. Introduce interface to allow in-memory caching of cipher keys. The
cache allowing mapping of "encryption domain" -> "base cipher id" ->
"derived cipher keys" (3D hash map). This cache interface will be used
by FDB processes participating in encryption to cache recently used
ciphers (performance optimization).
Testing:
1. Unit test to validate caching interface.
2. Update EncryptionOps simulation test to validate block cipher
operations.
2022-03-24 07:31:49 -07:00
Andrew Noyes
977fab2089
Make overwriteProxiesCount a function of conf keys ( #6645 )
...
* Improve assert diagnostics
* Make overwriteProxiesCount a deterministic function of conf keys
Previously, the final values for grv_proxy count, commit_proxy count,
and proxy count were derived from already-derived values from
overwriteProxiesCount. Instead, we should only look at the conf keys.
Also treat these keys as set to -1 if absent
* Add unit test
Illustrates how overwriteCommitProxy is not a function of conf keys.
Passes after change and fails before
2022-03-23 17:15:28 -07:00
Trevor Clinkenbeard
bdff100ef9
Merge pull request #6582 from sfc-gh-tclinkenbeard/global-tag-throttling2
...
Various `TagThrottler` enhancements
2022-03-23 12:54:30 -07:00
Evan Tschannen
4a085fc844
Merge pull request #6602 from apple/blob_integration
...
Blob integration
2022-03-23 12:02:43 -07:00
sfc-gh-tclinkenbeard
0726832e80
Merge remote-tracking branch 'origin/main' into global-tag-throttling2
2022-03-23 11:05:02 -07:00
Neethu Haneesha Bingi
790ef9ff36
Adding rocksdb compaction readahead size option.
2022-03-22 13:29:35 -07:00
Josh Slocum
f27475e2f4
Merge branch 'main' into blob_integration
2022-03-22 11:41:58 -05:00
sfc-gh-tclinkenbeard
a71099471b
Update copyright header dates
2022-03-21 13:36:23 -07:00
Jingyu Zhou
fb046c0665
Merge pull request #6615 from sfc-gh-tclinkenbeard/add-missing-copyright-headers
...
Add missing copyright headers
2022-03-21 13:17:15 -07:00
sfc-gh-tclinkenbeard
004e73b2f1
Add TagSet::toString method
2022-03-18 20:10:56 -07:00
sfc-gh-tclinkenbeard
6eaa20078f
Add ContainsRecommended boolean parameter
2022-03-18 18:12:50 -07:00
Josh Slocum
8205771e8f
ReplyPromiseStreams need synchronous disconnect to avoid change feed races
2022-03-18 17:05:12 -05:00
Yi Wu
548b36b381
fix typo
...
Co-authored-by: Steve Atherton <steve.atherton@snowflake.com>
2022-03-18 14:56:04 -07:00
Yi Wu
264b3811af
update the lag knob
2022-03-18 14:56:04 -07:00
Yi Wu
27df6df950
Redwood: config remap cleanup by size instead of versions
2022-03-18 14:56:04 -07:00
He Liu
f6ea04a4a1
Fixed uninitialized member issue.
2022-03-18 09:12:59 -07:00
Xiaoge Su
bd19845ef6
fixup! clang-format
2022-03-17 16:53:10 -07:00
Xiaoge Su
4bcc9d1168
Add guardian to some of the header files
...
Since #pragma once is not part of standard...
2022-03-17 16:53:10 -07:00
Josh Slocum
37e7c80f26
Merge branch 'main' into blob_integration
2022-03-17 18:45:42 -05:00
A.J. Beamon
b300dae8d8
Merge pull request #6612 from sfc-gh-ajbeamon/special-keys-use-raw-access
...
Avoid promoting transactions to have system key access when using special keys
2022-03-17 15:17:07 -07:00
A.J. Beamon
513e8887dd
Add a new C unit test that uses tenants. Fix simulation issue where resetting a transaction could cause in-flight operations to pick up the simulated default tenant for the next retry. Use the new list tenant JSON format in the tenant management test. Fix missing raw_access option in special key space correctness test.
2022-03-17 12:10:19 -07:00
A.J. Beamon
4079b9f23d
Lazily assign the default tenant in simulated tests so that we can enforce that raw access isn't used on tenant transactions. Update various tests to avoid using raw access or system key access after they have performed other operations on the transaction.
2022-03-17 12:10:19 -07:00
A.J. Beamon
d0dc756c6d
Allow disabling tenant mode in simulation. Fix a few bugs.
2022-03-17 12:10:18 -07:00
A.J. Beamon
05495908b8
Implement some tenant tests
2022-03-17 12:10:18 -07:00
A.J. Beamon
7d9148857f
Avoid promoting transactions to have system key access when using special keys
2022-03-17 08:33:53 -07:00
sfc-gh-tclinkenbeard
56026b5b6f
Merge remote-tracking branch 'origin/main' into global-tag-throttling2
2022-03-16 22:29:43 -07:00
sfc-gh-tclinkenbeard
c4bc35da09
Merge remote-tracking branch 'origin/main' into fix-includes
2022-03-16 22:29:18 -07:00
sfc-gh-tclinkenbeard
0e7dc83f25
Fix compilation issues with ModelInterface construction in configuration database code
2022-03-16 14:25:32 -07:00
sfc-gh-tclinkenbeard
58de6e22cc
Add BalanceOnRequests boolean parameter for ModelInterface
2022-03-16 14:25:32 -07:00
sfc-gh-tclinkenbeard
62f547ff6e
Add line before actorcompiler.h include (to prevent IDE from reordering includes)
2022-03-16 14:15:48 -07:00
sfc-gh-tclinkenbeard
71976e51c5
Merge remote-tracking branch 'origin/main' into global-tag-throttling2
2022-03-16 13:53:46 -07:00
sfc-gh-tclinkenbeard
faecd8a8f8
Merge remote-tracking branch 'origin/main' into global-tag-throttling2
2022-03-16 13:45:23 -07:00
Trevor Clinkenbeard
765e018afb
Merge pull request #6580 from sfc-gh-tclinkenbeard/const-smoother
...
Mark `Smoother::smooth*` methods `const`
2022-03-16 13:43:25 -07:00
sfc-gh-tclinkenbeard
1aabbb07d2
Add missing copyright headers
2022-03-16 11:25:02 -07:00
Trevor Clinkenbeard
10c536c700
Merge pull request #6435 from sfc-gh-ljoswiak/fixes/dynamic-knobs-release-readiness
...
Dynamic knobs improvements
2022-03-16 10:26:56 -07:00
sfc-gh-tclinkenbeard
66d71e107d
Move actorcompiler.h include to the end of includes
2022-03-16 00:09:16 -07:00
A.J. Beamon
a1dfd92a43
Fix merge conflict between tenants and checkpointing API. Avoid using .toString() in trace event details and rename (and move) a couple variables.
2022-03-15 14:18:06 -07:00
A.J. Beamon
7464c7747c
Merge pull request #6600 from sfc-gh-ajbeamon/add-tenant-support-back-to-mapped-range
...
Add tenant support back to mapped range requests
2022-03-15 13:46:32 -07:00
Josh Slocum
e8e69e7c56
Fix formatting
2022-03-15 15:15:16 -05:00
He Liu
c3a68d661e
Physical Shard Move ( #6264 )
...
Physical Shard Move part I: Checkpoint creation, transfer and restore.
2022-03-15 13:03:23 -07:00
Josh Slocum
ce5b567971
Cleanup the cleanup
2022-03-15 15:02:28 -05:00
A.J. Beamon
a04934465c
Add tenant support back to mapped range requests. Fix ACTOR warning.
2022-03-15 12:41:08 -07:00
A.J. Beamon
e8077b65e1
Merge pull request #6559 from sfc-gh-ajbeamon/fdb-tenant-client
...
Add client support for tenants
2022-03-15 12:40:17 -07:00
Josh Slocum
67eba5ec7c
Limiting DD Moves by destination SS.
2022-03-15 13:52:19 -05:00
A.J. Beamon
22faf8e5b3
Ignore tenants when setting watches on the metadataVersionKey or sending mutations or conflict ranges that use the metadataVersionKey.
2022-03-15 10:46:31 -07:00
Jingyu Zhou
e89ee7d5a0
Merge pull request #6589 from sfc-gh-tclinkenbeard/fix-typos
...
Fix typos
2022-03-15 10:10:23 -07:00
A.J. Beamon
8a5107af03
Fix a couple variable initialization issues in the tenant MVC implementation. Encode tenant prefixes with unicode escape sequences for the list command.
2022-03-15 09:23:30 -07:00
A.J. Beamon
a70adf9fcd
Convert cached ranges to relative ranges when using tenants.
2022-03-15 09:23:30 -07:00
A.J. Beamon
149ca44b9b
Disallow setting raw access on a transaction that has specified a tenant
2022-03-15 09:23:30 -07:00
