The fdb c shim build requires python3. Let's not allow python2 to get
used there.
Otherwise we can see something like this:
```
FAILED: bindings/c/libfdb_c.so.init.c bindings/c/libfdb_c.so.tramp.S
cd /home/anoyes/workspace/foundationdb/build/bindings/c && /usr/bin/python2.7 /home/anoyes/workspace/foundationdb/contrib/Implib.so/implib-gen.py --target aarch64 --outdir /home/anoyes/workspace/foundationdb/build/bindings/c --dlopen-callback=fdb_shim_dlopen_callback /home/anoyes/workspace/foundationdb/build/lib/libfdb_c.so
File "/home/anoyes/workspace/foundationdb/contrib/Implib.so/implib-gen.py", line 27
sys.stderr.write(f'{me}: warning: {msg}\n')
```
* Don't build fdb c shim with ubsan
This avoids duplicate symbols when linking. It doesn't really make sense
to assemble files with -fsanitize=undefined anyway, since it won't
insert instrumentation.
* Consolidate boost_asan with boost_target
* Fix ASAN build
Description
-diff-2: Fix Mac build issues
-diff-1: Address review comments
Patch addresses the issue where ASAN build failed after introducing
BlobGranule compression.
Testing
ASAN build
* proof of concept
* use code-probe instead of test
* code probe working on gcc
* code probe implemented
* renamed TestProbe to CodeProbe
* fixed refactoring typo
* support filtered output
* print probes at end of simulation
* fix missed probes print
* fix deduplication
* Fix refactoring issues
* revert bad refactor
* make sure file paths are relative
* fix more wrong refactor changes
Currently, we have code in different folders like `flow/` and `fdbrpc/`
that should remain isolated. For example, `flow/` files should not
include functionality from any other modules. `fdbrpc/` files should
only be able to include functionality from itself and from `flow/`.
However, when creating a shared library, the linker doesn't complain
about undefined symbols -- this only happens when creating an
executable. Thus, for example, it is possible to forward declare an
`fdbclient` function in an `fdbrpc` file and then use it, and nothing
will break (when it should, because this is illegal).
This change adds dummy executables for a few modules (`flow`, `fdbrpc`,
`fdbclient`) that will cause a linker error if there are included
symbols which the linker can't resolve.
* REST KmsConnector implementation
Description
diff-1: Address review comments.
Add utility interface to Platform namespace to
create and operate on tmpfile
diff-2: Address review comments
Link Boost::filesystem to CMake build process
Major changes includes:
1. Implement REST based KmsConnector implementation.
2. Salient features of the connector:
2.1. Two required configuration are:
a. Discovery KMS URLs - enable KMS discovery on bootstrap
b. Endpoint path configuration to construct URI to fetch/refresh
encryption keys
c. Configuration to provide "validationTokens" to connect with
external KMS. Patch implements file-based token validation scheme.
2.2. On startup, RESTKmsConnector discovers KMS Urls and caches
them in-memory. Extracts "validationTokens" based on input config.
2.3. Expose endpoints to allow fetch/refresh of encryption keys.
2.4. Defines JSON format to interact with external KMS - request &
response payload format.
3. Extend Platform namespace with an interface to create and operate on
tmp files.
4. Update Platform 'readFileBytes' and 'writeFileBytes' to leverage
fstream supported implementation.
NOTE: KMS URLs fetched after initial discovery will be persisted using
DynamicKnobs. It is TODO at the moment and shall be completed
once DynamicKnobs is feature complete
Testing
Unit test to validation following:
1. Parsing on "validation tokens" logic.
2. Construction and parsing of REST JSON request and response strings.
* Use execstack to test executable stack permissions
The output of readelf wasn't stable
* Tee output to stdout in pkg_tester "run" function
* Test compiling an app that includes fdb_c.h
Featuring both cmake and pkg_tester, since we intend to support those in
our packages.
* Fix packaging bug
The libdir in the pkg-config file in the packages was wrong
* Set default for USE_JEMALLOC initially in ConfigureCompiler
Instead of trying to change the value later on. This fixes the valgrind
build, which was previously incorrectly getting jemalloc involved.
* Check aligned_alloc result for null
And OOM if so - don't assert
* Check that we can allocate magazines with no internal fragmentation
We may want to do this so that the jemalloc heap profiler has some
knowledge of FastAlloc
* Populate TestFile field for noSim tests in TestHarness
* Remove handling for nonexistent "ActualRun"
