1
0
mirror of https://github.com/apple/foundationdb.git synced 2025-05-21 22:33:17 +08:00

21104 Commits

Author SHA1 Message Date
Jon Fu
7f3b51f31a initial commit to introduce tenant renaming to special keys 2022-07-13 15:59:52 -07:00
A.J. Beamon
712f40b727 Add support for tenant groups 2022-07-08 15:59:25 -07:00
Xiaoxi Wang
6be20d08ae
Merge pull request from sfc-gh-xwang/feature/dd-refactor-incremental
[DD Testability] move takeMoveKeysLock to DDTxnProcessor
2022-07-08 15:40:14 -07:00
He Liu
aa43956fbd
Sharded RocksDB ()
* Some fixes.

* Enabled ShardedRocksDB in IKeyValueStore.h

* Added unit test for ShardedRocks for \xff\xff key space read/write.

* Resolved comments.

* Return empty read results if the physical shards are not initialized.

Co-authored-by: He Liu <heliu@apple.com>
2022-07-08 15:17:38 -07:00
Bharadwaj V.R
aca44c8644
Increase frequency of RelocateShardStartSplit traces for better observability of splits () 2022-07-08 14:45:44 -07:00
Xiaoxi Wang
9cead35911 add contract comments 2022-07-08 14:19:14 -07:00
Xiaoxi Wang
f5a81154ef move takeMoveKeysLock to DDTxnProcessor 2022-07-08 14:11:31 -07:00
Jingyu Zhou
4103b16793 Fix a performance bug in Resolver
The erased bytes from recent state transactions was accidently set to 0 and
cause anyPopped to be always false. Thus, when totalStateBytes will never
decrease and can cause high latency between "Resolver.resolveBatch.Before" and
"Resolver.resolveBatch.AfterQueueSizeCheck", in hundreds of milliseconds. This
will cause high server side commit latency.
2022-07-08 09:58:28 -07:00
Vaidas Gasiunas
1e8feb9cb8
Generate a shim library for the FDB C API ()
* Adding sources of the Implib.so project

* Run C unit tests and API tests with the shim library

* Reuse compilation of C test binaries with and without shim library

* Resolve client library path from an environment variable

* Refactoring: Reusable module for downloading FDB binaries

* Testing client shim library with current version and last release version

* Tests for specifying client library over an environment variable

* Enable C shim library tests on ARM

* Restore the original path for including fdb_api.hpp
2022-07-08 16:28:35 +02:00
Marian Dvorsky
24d7e0d28b
Add WatchAndWait API client test workload ()
* Add WatchAndWait workload

* Reimplement without Notification

* Handle errors

* Finish comment

* Simplify the test by introducing commit(/*complete=*/false)

* Fix a race condition

* Refactor

* Remove unused capture

* Add assert for has_value

* Automatically adjust numClientThreads
2022-07-08 11:57:38 +02:00
He Liu
bc5bfaffda
Shard based move ()
* Shard based move.

* Clean up.

* Clear results on retry in getInitialDataDistribution.

* Remove assertion on SHARD_ENCODE_LOCATION_METADATA for compatibility.

* Resolved comments.

Co-authored-by: He Liu <heliu@apple.com>
2022-07-07 20:49:16 -07:00
Steve Atherton
e63cb431e1
Merge pull request from sfc-gh-satherton/storage-type-improvements
Add StorageEngine type to StorageMetrics trace events
2022-07-07 16:26:43 -07:00
A.J. Beamon
346d3c3265
Merge pull request from sfc-gh-ajbeamon/buggified-commit-helper
Add a helper to simulate a commit that can fail with unknown result or take a long time
2022-07-07 16:17:19 -07:00
Jingyu Zhou
5b3521f7f0 Add 7.1.12, 7.1.13 release notes 2022-07-07 15:13:01 -07:00
Markus Pilman
59cdb8ae7c
Merge pull request from goandrei/add_intel_compiler_support
Add Intel compiler support
2022-07-07 10:40:53 -06:00
Andrei Gorneanu
35f693b238 Disable ffast-math for Intel compiler 2022-07-07 17:36:26 +02:00
Andrei Gorneanu
3a0444b569 Merge remote-tracking branch 'upstream/main' into add_intel_compiler_support 2022-07-07 17:30:09 +02:00
Sreenath Bodagala
fa0c3249ca
Merge pull request from sbodagala/main
Make sequencer aware of the locality of the primary region
2022-07-07 14:00:10 +05:30
A.J. Beamon
c4b0f6eaae
Add an internal C API to support connection to a cluster using a connection string ()
* Add an internal C API to support memory connection records

* Track shared state in the client using a unique and immutable cluster ID from the cluster

* Add missing code to store the clusterId in the database state object

* Update some arguments to pass by const&
2022-07-07 10:12:49 +02:00
Steve Atherton
03df62e2fc Added StorageEngine detail to StorageMetrics trace event. 2022-07-07 00:10:04 -07:00
Steve Atherton
cf96495bb2 Hardcode StoreType as they cannot change. 2022-07-07 00:08:25 -07:00
Yao Xiao
f98d5ae9e9
[ShardedRocksDB] 5/N Generate and persist metadata for range mapping. () 2022-07-06 15:57:46 -07:00
A.J. Beamon
5852a55127
Merge pull request from sfc-gh-jfu/jfu-tenant-rename
Support Tenant Renaming
2022-07-06 15:08:04 -07:00
A.J. Beamon
ae6d6e0e72
Merge pull request from sfc-gh-jfu/jfu-mako-active-tenants
Introduce concept of "active" versus "total" tenants in mako
2022-07-06 14:23:11 -07:00
Jon Fu
2ecedc6c5d use refactored tenant entry encoding 2022-07-06 14:12:09 -07:00
Jon Fu
df1f108b78 Merge branch 'main' of github.com:apple/foundationdb into jfu-tenant-rename 2022-07-06 14:10:22 -07:00
Jon Fu
3bdbf0c109 fix compile error 2022-07-06 14:06:57 -07:00
Dan Lambright
bb99b59deb
Change VersionVectorTagUpdates to LatencySample () 2022-07-06 16:48:15 -04:00
Jon Fu
61b3e1f51e add comment about our tenant encoding 2022-07-06 13:16:24 -07:00
Jon Fu
af78a2ef5f move tr options to outside loop 2022-07-06 13:07:47 -07:00
Jon Fu
48b0f6e22b code review changes 2022-07-06 13:04:21 -07:00
A.J. Beamon
8897a03913 Allow for random delays in buggifiedCommit that don't cause timeouts or that occur before we throw commit_unknown_result. Add a delay(0) to avoid a nowait buggifiedCommit in the typical case. 2022-07-06 12:48:13 -07:00
A.J. Beamon
1e8225a19e
Merge pull request from sfc-gh-ajbeamon/add-subrange-support-to-tenant-special-keys
Update the tenant special keys submodule to support multiple sub-ranges
2022-07-06 10:33:14 -07:00
Sreenath Bodagala
a84112d30b - Propagate the locality of the primary region from ClusterController
to the sequencer, rather than deriving it from "MasterInterface" on
the sequencer (which is not correct).
2022-07-06 11:57:06 +00:00
Johannes M. Scheuermann
b9afde3d4c Fix permission in Dockerfile 2022-07-05 20:02:51 -07:00
Jingyu Zhou
2d9b854a9f
Merge pull request from sbodagala/main
Microbenchmark version vector serialization/deserialization code
2022-07-05 15:04:02 -07:00
Sreenath Bodagala
c366adb4b7 - Benchmark for version vector serialization/deserialization code 2022-07-05 16:37:49 +00:00
Marian Dvorsky
975978d938
Add getKey support to ApiCorrectnessWorkload ()
* Add getKey support to ApiCorrectnessWorkload

* Address review comments
2022-07-05 10:16:29 +02:00
Junhyun Shim
db449b80f7
Authz JWKS ()
* Add JsonWebKeySet parser/stringifier

* Update header directory

* Make JWKS parser correctness clean for OpenSSL 1.x

Add RSA keygen support

* Make JWKS parser correctness clean for OpenSSL 3.x

+extend unique_ptr for scoped destruction of OpenSSL objects

* Use PKey::{sign|verify}() in TokenSign

* Apply AutoCPointer to MkCert

* Apply Clang format

* JWKS::toStringRef() returns StringRef > Optional<StringRef>

* Fix Mac/Windows build error

* Fix incorrect fix of Mac build

* Fix filename in license comment for AutoCPointer.h

* Refactor complex C macros into function templates
2022-07-05 10:13:09 +02:00
Steve Atherton
c417df83f3
Merge pull request from sfc-gh-ajbeamon/more-generic-key-backed-types
Update key backed types to support multiple transaction types and different codecs
2022-07-01 14:37:02 -07:00
Markus Pilman
14f07944b3
Merge pull request from sfc-gh-jslocum/awscreds_cleanup
Cleaning up AWS creds
2022-07-01 14:51:46 -06:00
Josh Slocum
08ca0a9883
Merge pull request from sfc-gh-jslocum/cf_memory_perf
Change Feed Reading CPU optimization
2022-07-01 14:06:39 -05:00
Josh Slocum
bee8fdcb97 formatting 2022-07-01 13:51:17 -05:00
Josh Slocum
236ef93324 Cleaning up AWS creds 2022-07-01 13:30:53 -05:00
Dan Lambright
ea42e9f59c
Do not delay sending empty messages twice () 2022-07-01 14:22:53 -04:00
A.J. Beamon
9e659b62ee Fix idempotent spelling 2022-07-01 10:34:10 -07:00
A.J. Beamon
fd568a1179 Fix some missed ReadYourWritesTransaction instances 2022-07-01 10:24:06 -07:00
A.J. Beamon
62683b4140 Update key backed types to support a templated Codec. This required updating the existing Codec to convert from objects into byte strings through tuples. 2022-07-01 10:11:02 -07:00
A.J. Beamon
1dc78a3b3f Add support for different transaction types (Reference<ITransaction>, Reference<ReadYourWritesTransaction>) to key backed types 2022-07-01 10:02:39 -07:00
A.J. Beamon
aff21f062f Fix indentation and refactor prefix construction to be in forward order 2022-07-01 09:50:05 -07:00