3054 Commits

Author SHA1 Message Date
Junhyun Shim
e2a3fedfc7
Merge branch 'main' into features/authz 2022-07-27 00:08:57 +02:00
Josh Slocum
c32e1da908
Merge pull request #7673 from sfc-gh-jslocum/delta_files_v2
Sorted Delta Files
2022-07-26 16:04:55 -05:00
Xiaoxi Wang
7cf6037191 remove runAfter 2022-07-26 12:12:56 -07:00
Renxuan Wang
dc9599f2e9
Get network interfaces ready for https proxy. (#7556)
* Move HTTP from fdbclient/ to fdbrpc/.

* Move md5 and libb64 to contrib/.

* Get network interfaces ready for https proxy.

* Rebase
2022-07-25 17:08:32 -07:00
Andrew Noyes
83a317ef88
Disable guard pages for valgrind build. Closes #7610 (#7672) 2022-07-22 16:51:05 -07:00
Josh Slocum
ea9018460a cleanup and polish 2022-07-22 15:13:32 -05:00
Andrew Noyes
45c8a4e09d
Build fixes/improvements for sanitizers (#7657)
* Don't build fdb c shim with ubsan

This avoids duplicate symbols when linking. It doesn't really make sense
to assemble files with -fsanitize=undefined anyway, since it won't
insert instrumentation.

* Consolidate boost_asan with boost_target
2022-07-21 12:43:38 -07:00
Junhyun Shim
908b8faa88 Fix unmatched signatures for BoringSSL 2022-07-21 15:17:34 +02:00
Junhyun Shim
97880880cd Defer Simulation check in OSSL-IRandom binding until g_network is set 2022-07-21 14:52:02 +02:00
Junhyun Shim
f76021fe7b Apply Clang format 2022-07-21 12:12:48 +02:00
Junhyun Shim
e6fe4a62b4 Ensure DeterministicRandom gets used by OpenSSL exclusively in Simulation 2022-07-21 12:11:33 +02:00
A.J. Beamon
190711ccd8
Merge pull request #7622 from sfc-gh-ajbeamon/eat-and-connection-string-changes
Add some helper functions to ClusterConnectionString and an improvement to StringRef.eat
2022-07-20 16:10:24 -07:00
A.J. Beamon
4d036ae339
Merge pull request #7626 from sfc-gh-ajbeamon/tenant-metadata-change
Some changes to the tenant metadata
2022-07-20 14:50:00 -07:00
Ata E Husain Bohra
9a3e88df46
Fix ASAN build (#7629)
* Fix ASAN build

Description
  -diff-2: Fix Mac build issues
  -diff-1: Address review comments

Patch addresses the issue where ASAN build failed after introducing
BlobGranule compression.

Testing

ASAN build
2022-07-20 14:14:46 -07:00
Junhyun Shim
b521dc2f58 Bind DeterministicRandom to OpenSSL RNG 2022-07-20 16:03:09 +02:00
Junhyun Shim
0874e553cd Place generateRandomData() under {I|Deterministic}Random 2022-07-20 13:21:11 +02:00
A.J. Beamon
537ceff8ac Remove the ability to configure a tenant subspace. Rename the prefixes used for tenant metadata. 2022-07-19 14:32:05 -07:00
Markus Pilman
1de37afd52
Make TEST macros C++ only (#7558)
* proof of concept

* use code-probe instead of test

* code probe working on gcc

* code probe implemented

* renamed TestProbe to CodeProbe

* fixed refactoring typo

* support filtered output

* print probes at end of simulation

* fix missed probes print

* fix deduplication

* Fix refactoring issues

* revert bad refactor

* make sure file paths are relative

* fix more wrong refactor changes
2022-07-19 13:15:51 -07:00
A.J. Beamon
7324f28441 Add the ability for StringRef.eat to return whether a separator was found. Add a serialize function and equality operators to ClusterConnectionString. Remove unused variable. 2022-07-19 11:55:58 -07:00
Renxuan Wang
570db7a7d2
Fix processDiskReadSeconds and processDiskWriteSeconds. (#7608) 2022-07-18 13:47:45 -07:00
Junhyun Shim
8b70c7050d Apply RTLD_NODELETE for USE_SANITIZER builds
This allows their stack trace to be symbolized in CI/Jenkins XSAN runs
2022-07-18 19:25:12 +02:00
Andrew Noyes
989e323991
Attempt to fix windows build (#7600)
* Attempt to fix windows build

The windows build is complaining that some symbols in flow are
duplicate, which seems fair since we're currently compiling all the flow
sources and _also_ linking to flow for flowlinktest. This change makes
it so that we only link flow, and don't compile the flow source files
again.

* Remove source files from fdbclient and fdbrpc link tests
2022-07-18 08:59:03 -07:00
Ata E Husain Bohra
f6f117592d BlobFile Encryption and compression support
- Limit verbose logging under DEBUG_MACRO
 - Update/Add code documentation

Description

Testing
2022-07-14 17:04:14 -07:00
Ata E Husain Bohra
24b2de8de8 BlobFile Encryption and compression support
Description

Testing
2022-07-14 17:04:14 -07:00
Markus Pilman
2edbcf2c65
Merge pull request #44 from apple/main
Merge main
2022-07-12 07:51:22 -06:00
Markus Pilman
fa9e623fc1
Merge pull request #7459 from sfc-gh-ljoswiak/features/module-link-tests
Add test executables to catch missing symbols in modules
2022-07-11 17:31:11 -06:00
Junhyun Shim
61f3d14ec6 Add TokenCache test
Add function authz::jwt::TokenRef::toStringRef() for token tracing
Add StringRef::toStringView()
2022-07-11 13:42:41 +02:00
He Liu
bc5bfaffda
Shard based move (#6981)
* Shard based move.

* Clean up.

* Clear results on retry in getInitialDataDistribution.

* Remove assertion on SHARD_ENCODE_LOCATION_METADATA for compatibility.

* Resolved comments.

Co-authored-by: He Liu <heliu@apple.com>
2022-07-07 20:49:16 -07:00
A.J. Beamon
346d3c3265
Merge pull request #7487 from sfc-gh-ajbeamon/buggified-commit-helper
Add a helper to simulate a commit that can fail with unknown result or take a long time
2022-07-07 16:17:19 -07:00
Markus Pilman
56541167c2 send token with tenant info object 2022-07-06 20:58:17 -06:00
Lukas Joswiak
618f8455c4 Add test executables to catch missing symbols
Currently, we have code in different folders like `flow/` and `fdbrpc/`
that should remain isolated. For example, `flow/` files should not
include functionality from any other modules. `fdbrpc/` files should
only be able to include functionality from itself and from `flow/`.
However, when creating a shared library, the linker doesn't complain
about undefined symbols -- this only happens when creating an
executable. Thus, for example, it is possible to forward declare an
`fdbclient` function in an `fdbrpc` file and then use it, and nothing
will break (when it should, because this is illegal).

This change adds dummy executables for a few modules (`flow`, `fdbrpc`,
`fdbclient`) that will cause a linker error if there are included
symbols which the linker can't resolve.
2022-07-06 14:49:33 -07:00
A.J. Beamon
8897a03913 Allow for random delays in buggifiedCommit that don't cause timeouts or that occur before we throw commit_unknown_result. Add a delay(0) to avoid a nowait buggifiedCommit in the typical case. 2022-07-06 12:48:13 -07:00
Junhyun Shim
db449b80f7
Authz JWKS (#7519)
* Add JsonWebKeySet parser/stringifier

* Update header directory

* Make JWKS parser correctness clean for OpenSSL 1.x

Add RSA keygen support

* Make JWKS parser correctness clean for OpenSSL 3.x

+extend unique_ptr for scoped destruction of OpenSSL objects

* Use PKey::{sign|verify}() in TokenSign

* Apply AutoCPointer to MkCert

* Apply Clang format

* JWKS::toStringRef() returns StringRef > Optional<StringRef>

* Fix Mac/Windows build error

* Fix incorrect fix of Mac build

* Fix filename in license comment for AutoCPointer.h

* Refactor complex C macros into function templates
2022-07-05 10:13:09 +02:00
Yi Wu
364644673f
Support TLog encryption in commit proxy (#6942)
This PR add support for TLog encryption through commit proxy. The encryption is done on per-mutation basis. As CP writes mutations to TLog, it inserts encryption header alongside encrypted mutations. Storage server (and other consumers of TLog such as storage cache and backup worker) decrypts the mutations as they peek TLog.
2022-06-29 14:21:05 -07:00
sfc-gh-tclinkenbeard
2134fa9a0c Merge remote-tracking branch 'origin/main' into global-tag-throttling3 2022-06-28 19:51:41 -07:00
A.J. Beamon
55404d0cd8 Add a helper to simulate a commit that can fail with unknown result or take a long time 2022-06-28 16:41:57 -07:00
Markus Pilman
9d16260822 Merge remote-tracking branch 'origin/main' into features/authz 2022-06-28 13:13:51 -06:00
sfc-gh-tclinkenbeard
086e4bff06 Merge remote-tracking branch 'origin/main' into global-tag-throttling3 2022-06-28 10:18:13 -07:00
Markus Pilman
20e7371036 some Linux fixes 2022-06-28 08:51:13 -06:00
Markus Pilman
d31fb30f57 fdbclient compiling 2022-06-27 18:31:05 -06:00
Markus Pilman
88af6b403f fdbrpc compiling 2022-06-27 18:11:55 -06:00
Markus Pilman
03d913a1de Flow compiling 2022-06-27 17:05:55 -06:00
Markus Pilman
3aaae9c521 Merge remote-tracking branch 'origin/main' into features/authz 2022-06-27 11:07:14 -06:00
Markus Pilman
e35633149c
Merge pull request #7447 from sfc-gh-mpilman/bugfixes/cmake-disallow-illegal-cross-module-includes
enforce include modularization in cmake
2022-06-24 22:42:18 -06:00
Markus Pilman
a47ed89018 Linux fixes and addressed review comments 2022-06-23 20:52:13 -06:00
Markus Pilman
de48e90276 fdbserver is now compiling 2022-06-23 18:45:26 -06:00
Markus Pilman
8af056e7b0 fdbclient now compiling 2022-06-23 18:05:36 -06:00
Markus Pilman
10e478dfc3 Flow is compiling 2022-06-23 16:35:19 -06:00
Markus Pilman
d35445a868 enforce include modularization in cmake 2022-06-23 14:37:35 -06:00
sfc-gh-tclinkenbeard
840dac1fa3 Merge remote-tracking branch 'origin/main' into global-tag-throttling3 2022-06-22 22:17:33 -07:00