24390 Commits

Author SHA1 Message Date
Vaidas Gasiunas
4a178fe658 get_client_status: fix formatting in comments 2023-01-16 17:36:32 +01:00
Vaidas Gasiunas
46178f4c74 get_client_report: commenting DatabaseState::getClientStatus 2023-01-16 16:50:17 +01:00
Vaidas Gasiunas
d064487550 client_status_report: Change variable names 2023-01-16 15:17:37 +01:00
Vaidas Gasiunas
2ebcc49465 get_client_status: a comment giving a more precise description 2023-01-16 15:09:11 +01:00
Vaidas Gasiunas
5b02755396 get_client_status: document the schema of the returned json 2023-01-16 15:07:43 +01:00
Vaidas Gasiunas
e152fb28af client_status_report: report cluster id from the DatabaseContext only 2023-01-16 14:57:46 +01:00
Vaidas Gasiunas
d43b4e5777 get_client_status: remove redundant call to getRecentClosedPeers 2023-01-16 14:01:01 +01:00
Vaidas Gasiunas
735bfbc05f client_config_tester: Add a comment describing its contract 2023-01-16 13:03:28 +01:00
Vaidas Gasiunas
64d81a8c95 client_config_tester: Adding missing break in switch case 2023-01-16 12:28:35 +01:00
Vaidas Gasiunas
962cd6efcd get_client_status: determine the health status 2023-01-12 18:46:04 +01:00
Vaidas Gasiunas
205466e04c get_client_report: report storage servers 2023-01-12 15:15:07 +01:00
Vaidas Gasiunas
8d734fba85 get_client_status: report database connection status 2023-01-11 17:43:35 +01:00
Vaidas Gasiunas
265c8f7f78 get_client_status: Fix compiler warning 2023-01-10 14:31:04 +01:00
Vaidas Gasiunas
6162fc333c get_client_status: Provide more MVC database status info, test client versions 2023-01-09 19:36:39 +01:00
Vaidas Gasiunas
f14f2ccc0d Introducing API function get_client_status; Exposing and testing database initialization state 2023-01-09 16:09:58 +01:00
Yi Wu
845cc62a39
Redwood: fix tree height overgrowth with per-tenant encryption (#9020)
* Fix Redwood tree height overgrowth when EaR and tenant page split are enabled, by removing the buildNewSubtree() logic.
* Fixing incorrect page upper bound for the last page created by writePages() without the buildNewSubtree() logic.
* Enable tenant page split if encryption mode is domain-aware encryption.
* Related test fixes:
  - In simulation, pass encryption mode to storage/Redwood via knobs. This is a workaround to enable testing with Redwood encryption before we correctly pass the encryption mode via db config. Also temporarily disable tenant page split for restart tests.
  - Disable raw access in FuzzApiCorrectness test if domain-aware encryption is enabled, to avoid test timeout
  - Disable encryption for DrUpgradeRestart test, which is likely to fail due to a rare EKP deadlock issue blocking recovery. Will re-enable after the deadlock issue is fixed.
2023-01-06 15:56:37 -08:00
A.J. Beamon
05bbb7c840
Merge pull request #9015 from sfc-gh-ajbeamon/expand-optional-erroror-map
Add overloads of the Optional::map and ErrorOr::map functions that allow easy member access
2023-01-06 15:44:15 -08:00
A.J. Beamon
a5f882c0ac Merge branch 'main' into expand-optional-erroror-map 2023-01-06 14:29:50 -08:00
Nim Wijetunga
10ccaa1ee5
remove client info encryption state (#9096) 2023-01-06 17:14:06 -05:00
Yi Wu
5d6ba48da0
Fix /redwood/correctness/EnforceEncodingType unit test (#9095)
The unit test intentionally cause unexpected_encoding_type being thrown, which would hit a simulation only assert failure. Disabling that assert in this case.
2023-01-06 14:13:59 -08:00
Zhe Wu
a444bcdf39
Merge pull request #9100 from halfprice/zhewu/satellite-disconnection-trigger-recovery
Disconnection to satellite TLog should trigger recovery in gray failure detection
2023-01-06 14:09:54 -08:00
Ankita Kejriwal
72121204d6
Merge pull request #9057 from sfc-gh-anoyes/anoyes/idempotency-docs
Add initial documentation for automatic idempotency
2023-01-06 13:43:09 -08:00
Zhe Wu
4f7f5ec4d8 Merge remote-tracking branch 'upstream/main' into zhewu/satellite-disconnection-trigger-recovery 2023-01-06 11:28:15 -08:00
He Liu
33a43cde79
Fixed error type. (#9101) 2023-01-06 10:45:58 -08:00
Zhe Wu
6aaf5af75d Disconnection to satellite TLog should trigger recovery in gray failure detection 2023-01-05 22:23:02 -08:00
A.J. Beamon
b69bec758d Merge branch 'main' into expand-optional-erroror-map 2023-01-05 09:30:33 -08:00
Nim Wijetunga
21611761bd
Backup uses DB Config (#8941)
* add encryption db config

* address pr comments

* address pr comments

* add comments

* remove knobs from backup

* remove import

* cp uses db config

* modify simulated cluster

* remove includes

* fix tests

* fix tests

* modify comment

* add encryption enabled method

* change error to warn

* Trigger Build

* Trigger Build

* Trigger Build
2023-01-04 22:43:51 -05:00
Xiaoxi Wang
8266f52dea
Merge pull request #9012 from sfc-gh-xwang/feature/main/wiggleDelay
Persist accumulated wiggle delay
2023-01-04 16:14:09 -08:00
A.J. Beamon
62604bcc28 Merge branch 'main' into expand-optional-erroror-map 2023-01-04 15:46:23 -08:00
A.J. Beamon
e72a1332af Use const& for functor arguments 2023-01-04 14:56:09 -08:00
Hui Liu
46d92bbf3f
Merge pull request #8984 from sfc-gh-huliu/restoretest
Add correctness test for blob restore
2023-01-04 14:43:49 -08:00
He Liu
34b3172e34
Improved SHARD_ENCODE_LOCATION_METADATA migration. (#9077)
* Improved SHARD_ENCODE_LOCATION_METADATA migration.

* Cleanup.

* Cancel itself if a data move finds a conflicting data move. Fixed
transaction reset issue.

* Cancel data move in a retry loop to avoid corrupted mutations.

Co-authored-by: He Liu <heliu@apple.com>
2023-01-04 12:57:33 -08:00
Hui Liu
e3bf79cf71 Add correctness test for blob restore 2023-01-04 11:10:34 -08:00
Xiaoxi Wang
fc89287ef1 add DataForDc in StorageWiggleData; rename KeyBackedConfig to KeyBackedStruct 2023-01-04 09:02:20 -08:00
dependabot[bot]
b6dd94c408 Bump setuptools from 65.3.0 to 65.5.1 in /documentation/sphinx
Bumps [setuptools](https://github.com/pypa/setuptools) from 65.3.0 to 65.5.1.
- [Release notes](https://github.com/pypa/setuptools/releases)
- [Changelog](https://github.com/pypa/setuptools/blob/main/CHANGES.rst)
- [Commits](https://github.com/pypa/setuptools/compare/v65.3.0...v65.5.1)

---
updated-dependencies:
- dependency-name: setuptools
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-04 10:52:10 -06:00
Jingyu Zhou
1e98e4facd
Merge pull request #9089 from jzhou77/fix
Fix use after free bug
2023-01-03 15:01:47 -08:00
sfc-gh-tclinkenbeard
1efe06da20 Move SingletonRecruitThrottler to SingletonRoles.h 2023-01-03 14:08:09 -08:00
sfc-gh-tclinkenbeard
68547a2dbd Remove dead code from ClusterController.actor.cpp 2023-01-03 14:08:09 -08:00
sfc-gh-tclinkenbeard
9e9415eff0 Move singleton role logic into its own file 2023-01-03 14:08:09 -08:00
sfc-gh-tclinkenbeard
d7e8c6d7f9 Improve const-correctness in ClusterController.actor.h 2023-01-03 14:08:09 -08:00
Nim Wijetunga
cd01c3a711
Blob Worker uses Encryption DB Config (#8934)
* add encryption db config

* address pr comments

* address pr comments

* add comments

* cp uses db config

* remove includes

* fix tests

* fix tests

* modify comment

* add encryption enabled method

* Blob worker uses encryption db config

* fix simulation encrypt mode choice

* Trigger Build

* address pr comments

* change comment

* formatting fixes
2023-01-03 16:56:02 -05:00
Jingyu Zhou
52a32c837f Fix use after free bug
Line 948 hit this bug where id was free'ed at line 946
946	pendingDeletionShards.pop_front();
948	physicalShards.erase(id);
2023-01-03 13:48:18 -08:00
sfc-gh-tclinkenbeard
a408a90cd0 Removed unnecessary rare annotations 2023-01-03 11:17:07 -08:00
Jingyu Zhou
36d9f6fe46
Merge pull request #9063 from apple/dependabot/pip/tests/authorization/cryptography-38.0.3
Bump cryptography from 37.0.4 to 38.0.3 in /tests/authorization
2023-01-03 11:06:29 -08:00
Mohamed Oulmahdi
6472282e07 Fix Windows build broken by #8904 2023-01-03 12:45:59 -06:00
Yi Wu
17fdbc46a5
EaR: Add page checksum to Redwood pages in no-auth mode (#8965)
Previously with EaR we always enable authentication (e.g. we encrypt Redwood pages). The authentication is a form of checksum, so dedicated page checksum was not needed. This PR adds back xxhash page checksum when authentication is disabled. Also change the knob to default disable authentication.
2023-01-03 10:30:07 -08:00
Xiaoxi Wang
5de0c87654 add comments; remove unnecessary actor suffix; code format 2023-01-02 23:59:46 -08:00
Xiaoxi Wang
bbcb3cc018 extract KeyBackedConfig, StorageWiggleData class; solve template resolution problem; solve MV txn and native api conflict by splitting RunTransaction file 2023-01-02 23:34:39 -08:00
Andrew Noyes
f0cb73a941 Address review comments 2022-12-28 13:29:57 -08:00
neethuhaneesha
c9fd51d2ec
Adding rocksdb read latency metrics. (#9036) 2022-12-22 13:00:43 -08:00