apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-06-01 18:56:00 +08:00
Code Issues Packages Projects Releases Wiki Activity
21,098 Commits 35 Branches 318 Tags
bb05c83e19
Commit Graph

5170 Commits

Author SHA1 Message Date
He Liu
bc5bfaffda
Shard based move () 
* Shard based move.

* Clean up.

* Clear results on retry in getInitialDataDistribution.

* Remove assertion on SHARD_ENCODE_LOCATION_METADATA for compatibility.

* Resolved comments.

Co-authored-by: He Liu <heliu@apple.com>
 2022-07-07 20:49:16 -07:00
Steve Atherton
e63cb431e1
Merge pull request from sfc-gh-satherton/storage-type-improvements 
Add StorageEngine type to StorageMetrics trace events
 2022-07-07 16:26:43 -07:00
A.J. Beamon
346d3c3265
Merge pull request from sfc-gh-ajbeamon/buggified-commit-helper 
Add a helper to simulate a commit that can fail with unknown result or take a long time
 2022-07-07 16:17:19 -07:00
A.J. Beamon
c4b0f6eaae
Add an internal C API to support connection to a cluster using a connection string () 
* Add an internal C API to support memory connection records

* Track shared state in the client using a unique and immutable cluster ID from the cluster

* Add missing code to store the clusterId in the database state object

* Update some arguments to pass by const&
 2022-07-07 10:12:49 +02:00
Steve Atherton
cf96495bb2 Hardcode StoreType as they cannot change. 2022-07-07 00:08:25 -07:00
Jon Fu
2ecedc6c5d use refactored tenant entry encoding 2022-07-06 14:12:09 -07:00
Jon Fu
df1f108b78 Merge branch 'main' of github.com:apple/foundationdb into jfu-tenant-rename 2022-07-06 14:10:22 -07:00
Jon Fu
3bdbf0c109 fix compile error 2022-07-06 14:06:57 -07:00
Jon Fu
48b0f6e22b code review changes 2022-07-06 13:04:21 -07:00
A.J. Beamon
1e8225a19e
Merge pull request from sfc-gh-ajbeamon/add-subrange-support-to-tenant-special-keys 
Update the tenant special keys submodule to support multiple sub-ranges
 2022-07-06 10:33:14 -07:00
Jingyu Zhou
2d9b854a9f
Merge pull request from sbodagala/main 
Microbenchmark version vector serialization/deserialization code
 2022-07-05 15:04:02 -07:00
Sreenath Bodagala
c366adb4b7 - Benchmark for version vector serialization/deserialization code 2022-07-05 16:37:49 +00:00
Steve Atherton
c417df83f3
Merge pull request from sfc-gh-ajbeamon/more-generic-key-backed-types 
Update key backed types to support multiple transaction types and different codecs
 2022-07-01 14:37:02 -07:00
Markus Pilman
14f07944b3
Merge pull request from sfc-gh-jslocum/awscreds_cleanup 
Cleaning up AWS creds
 2022-07-01 14:51:46 -06:00
Josh Slocum
bee8fdcb97 formatting 2022-07-01 13:51:17 -05:00
Josh Slocum
236ef93324 Cleaning up AWS creds 2022-07-01 13:30:53 -05:00
A.J. Beamon
9e659b62ee Fix idempotent spelling 2022-07-01 10:34:10 -07:00
A.J. Beamon
fd568a1179 Fix some missed ReadYourWritesTransaction instances 2022-07-01 10:24:06 -07:00
A.J. Beamon
62683b4140 Update key backed types to support a templated Codec. This required updating the existing Codec to convert from objects into byte strings through tuples. 2022-07-01 10:11:02 -07:00
A.J. Beamon
1dc78a3b3f Add support for different transaction types (Reference<ITransaction>, Reference<ReadYourWritesTransaction>) to key backed types 2022-07-01 10:02:39 -07:00
Sreenath Bodagala
037c06a688 Merge remote-tracking branch 'apple-upstream/main' 2022-07-01 04:51:17 +00:00
A.J. Beamon
2f67328a0c Update the tenant special keys submodule to support multiple sub-ranges. This will enable future work that allows configuring tenants at the same time as creating them. 2022-06-30 15:03:37 -07:00
Josh Slocum
48bcf6eaba
parameterizing min split size instead of hardcoding it to min_shard_bytes () 2022-06-30 12:28:22 -07:00
Xiaoxi Wang
e85e7794c2
Merge pull request from sfc-gh-xwang/features/ppw-pause-state 
Add timeout handler for storage wiggler status fetcher
 2022-06-30 11:23:29 -07:00
Jingyu Zhou
d60cab788e
Merge pull request from jzhou77/main 
Add pipelining for secondary queries in index prefetch
 2022-06-30 10:27:29 -07:00
A.J. Beamon
76a147a104
Merge pull request from sfc-gh-ajbeamon/refactor-tenant-map-encoding 
Refactor how tenant map entries are encoded and decoded
 2022-06-30 09:48:38 -07:00
Dan Lambright
98b18e3a18
Remove code obsoleted by commit c48d5690 () 2022-06-30 12:16:23 -04:00
Xiaoxi Wang
801c1e8be2 Schema fix 2022-06-29 23:07:04 -07:00
Xiaoxi Wang
b51e9430a5 fix timeout buggify 2022-06-29 22:46:50 -07:00
Jingyu Zhou
ba5e2ed72d Add batching for secondary queries in index prefetch 
This is to reduce excessive load being issued concurrently, controlled by knob
MAX_PARALLEL_QUICK_GET_VALUE.
 2022-06-29 21:17:11 -07:00
Xiaoxi Wang
1b0780398f Merge branch 'main' of https://github.com/apple/foundationdb into features/ppw-pause-state 2022-06-29 21:03:47 -07:00
Xiaoxi Wang
570dfd6760 add timeout handler 2022-06-29 15:30:37 -07:00
Markus Pilman
8123070c2a move rapidxml to its own target 2022-06-29 16:10:14 -06:00
Yi Wu
364644673f
Support TLog encryption in commit proxy () 
This PR add support for TLog encryption through commit proxy. The encryption is done on per-mutation basis. As CP writes mutations to TLog, it inserts encryption header alongside encrypted mutations. Storage server (and other consumers of TLog such as storage cache and backup worker) decrypts the mutations as they peek TLog.
 2022-06-29 14:21:05 -07:00
Jon Fu
5e7bb0aa21 Merge branch 'main' of github.com:apple/foundationdb into jfu-tenant-rename 2022-06-29 13:20:19 -07:00
Chaoguang Lin
29f98f3654
Avoid duplicate snapshot on one process if it serves as multiple roles () 
* Fix comments

* Add simulation value for SERVER_KNOBS->SNAP_CREATE_MAX_TIMEOUT

* A work version with correctness clean

* Remove unnecessay comments; debugging symbols

* Only check secondary address for coordinators, same as before

* Change the trace to SevError and remove the ASSERT(false)

* Remove TLogSnapRequest handling on TlogServer, which is changed to use WorkerSnapRequest

* Add retry for network failures

* Add retry limit for network failures; still allow duplicate snapshots on processes are both tlog and storage to avoid race

* Add retry limit as a knob and make backoff exponentail

* Add getDatabaseConfiguration(Transaction* tr)

* revert back to send request for each role once

* update some comments
 2022-06-29 11:23:07 -07:00
A.J. Beamon
26b35c07cd Refactor how tenant map entries are encoded and decoded. Add a specific version to the encoding that matches the version used when this feature was introduced (and the only version in which it was used). 2022-06-29 10:58:58 -07:00
Trevor Clinkenbeard
db769667ae
Merge pull request from sfc-gh-tclinkenbeard/global-tag-throttling3 
Create Global Tag Throttler
 2022-06-29 10:06:00 -07:00
Sreenath Bodagala
a48008fc9b - Address a review comment 2022-06-29 10:16:58 +00:00
Sreenath Bodagala
c74710f6d5 - Address a review comment 
- Optiize out the code in DatabaseContext::getLatestCommitVersions()
in the case where the feature is turned off.
 2022-06-29 10:13:08 +00:00
sfc-gh-tclinkenbeard
2134fa9a0c Merge remote-tracking branch 'origin/main' into global-tag-throttling3 2022-06-28 19:51:41 -07:00
A.J. Beamon
55404d0cd8 Add a helper to simulate a commit that can fail with unknown result or take a long time 2022-06-28 16:41:57 -07:00
sfc-gh-tclinkenbeard
be9a2002c3 Merge remote-tracking branch 'origin/main' into fix-knob-typo 2022-06-28 15:58:18 -07:00
Jon Fu
5fee5e1a73 add renameTenant back to TenantAPI after rebase 2022-06-28 11:54:03 -07:00
sfc-gh-tclinkenbeard
086e4bff06 Merge remote-tracking branch 'origin/main' into global-tag-throttling3 2022-06-28 10:18:13 -07:00
Markus Pilman
f9f522092d move new headers 2022-06-28 10:51:32 -06:00
Markus Pilman
fc2f695b15 Merge remote-tracking branch 'origin/main' into features/cmake-refactor-dirs 2022-06-28 10:41:07 -06:00
A.J. Beamon
fcd91f676a
Merge pull request from sfc-gh-ajbeamon/tenant-reorganization 
Tenant code refactoring
 2022-06-28 09:16:19 -07:00
Markus Pilman
1bdbd89839 Merge remote-tracking branch 'origin/main' into features/cmake-refactor-dirs 2022-06-28 08:52:03 -06:00
Sreenath Bodagala
0da13a7797 - Optimize out the version vector specific code (on the client) 
when the version vector feature is disabled.
 2022-06-28 09:17:53 +00:00