Nim Wijetunga
6f37f55917
Restore System Keys First in Backup/Restore Workloads ( #8475 )
...
* system key restore ordering
* restore system keys before regular data
* atomic restore backup fix
* change testing
* fix compile error
* fix compile issue
* fix compile issues
* Trigger Build
* only split restore if encryption is enabled
* revert knob changes
* Update fdbserver/workloads/AtomicSwitchover.actor.cpp
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
* Update fdbserver/workloads/AtomicSwitchover.actor.cpp
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
* Update fdbserver/workloads/BackupCorrectness.actor.cpp
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
* Update fdbserver/workloads/AtomicRestore.actor.cpp
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
* add todo
* strengthen check
* seperate system restore for atomic restore
* address pr comments
* address pr comments
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-10-26 09:38:27 -07:00
Steve Atherton
344290fdf1
IO buffer lifetime fix, buf must stay alive until the file read completes.
2022-10-23 16:28:31 -07:00
Jingyu Zhou
57fc39d5b7
Make decodeKVPairs a non-actor function
2022-10-20 19:34:44 -07:00
Nim Wijetunga
d439bc1e6e
TenantEntryCache Watch Based Refresh ( #8399 )
...
* tenant modification changekey
* address pr comments
* change backup to use watch based tenant cache
* format
* address pr comments
* trigger build
* add todo
* check tenants disabled
* trigger build
* trigger build
* address pr comments
* address pr comments
* trigger build
2022-10-18 19:05:07 -07:00
Nim Wijetunga
8ca6397e97
Enable Backup Tests for Encrypted Snapshots ( #8419 )
...
* enable encrypted snapshot backup tests
* address pr comments
* remove tenant cache when decoding
* remove client knob
2022-10-12 10:45:30 -07:00
Markus Pilman
ea1325a552
Merge pull request #8319 from sfc-gh-tclinkenbeard/add-rare-code-probe-annotation
...
Add `rare` code probe decoration
2022-10-07 09:39:00 -06:00
Markus Pilman
550488b020
Merge remote-tracking branch 'origin/main' into bugfixes/open-for-ide
...
# Conflicts:
# bindings/c/CMakeLists.txt
# fdbclient/include/fdbclient/GetEncryptCipherKeys.actor.h
# fdbserver/BackupWorker.actor.cpp
# fdbserver/BlobWorker.actor.cpp
# fdbserver/CommitProxyServer.actor.cpp
# fdbserver/KeyValueStoreMemory.actor.cpp
# fdbserver/StorageCache.actor.cpp
# fdbserver/include/fdbserver/GetEncryptCipherKeys.actor.h
# fdbserver/storageserver.actor.cpp
# fdbserver/workloads/PhysicalShardMove.actor.cpp
# flow/CMakeLists.txt
2022-10-04 18:27:48 -06:00
A.J. Beamon
55b880432e
Merge pull request #8379 from sfc-gh-ajbeamon/literal-string-ref-removal
...
LiteralStringRef removal
2022-10-04 09:41:42 -07:00
Nim Wijetunga
778cb63f71
Ensure nullptr is not passed to memcpy ( #8376 )
...
* fix
* fix
* Trigger Build
2022-09-30 17:28:43 -07:00
A.J. Beamon
111bf9e34d
Add macros for literal string ref versions of __FUNCTION__ and __FILE__
2022-09-30 16:05:47 -07:00
Nim Wijetunga
232e23c87e
Snapshot Backup Encryption ( #8095 )
...
* inital commit
* add encryption support
* refactor passing encryption keys
* disable encryption codepath
* pass enable encryption cli arg for fdbbackup
* change encryption key passing
* only enable encryption for backup if tenant mode is required
* revert server knobs
* ignore json trace files
* --amend
* integrate ekp
* add comments
* comments
* comments
* revert knobs
* fix bug
* modify encrypted backup logic
* use common prefix
* check boundaries between begin and end key
* return system encrypt domain
* modify
* add tenant entry cache to backup
* slight revert
* format
* Trigger Build
* format
* use tenant name to fetch keys
* Trigger Build
* Trigger Build
* merge
* fix getting ekp
* fix bug
* address pr comments
* Trigger Build
* Trigger Build
* remove faulty check
2022-09-29 14:45:47 -07:00
A.J. Beamon
3353103d9d
Fix filtering of potential backup mutations in commit proxy and backup worker; add code probe to ensure we are testing default backup sharing and add some tests to hit it
2022-09-27 15:25:30 -07:00
sfc-gh-tclinkenbeard
985958c260
Add rare code probe decoration
2022-09-25 15:28:32 -07:00
A.J. Beamon
a7c200b16c
Add support for sharing mutation streams between a default backup and DR
2022-09-22 11:36:46 -07:00
A.J. Beamon
fda0d7223d
Update backup to include system key ranges needed for tenants. Run simulated backup tests with tenants.
2022-09-22 10:00:13 -07:00
Nim Wijetunga
eadb769cfa
Encrypt Backup Mutation Log ( #8159 )
...
* encrypt backup mutation log
* format
* address pr comments
* format
* fix bug
* revert knobs
* address pr comments
2022-09-20 15:43:39 -07:00
A.J. Beamon
4fd64630e8
Convert literal string ref instances to use _sr suffix
2022-09-19 11:35:58 -07:00
Nim Wijetunga
91fb7c72c8
remove code probes
2022-09-14 10:40:35 -07:00
A.J. Beamon
410f27412b
Merge pull request #7620 from sfc-gh-ajbeamon/make-tuple
...
Add a Tuple::makeTuple function to easily construct a tuple
2022-07-20 17:09:10 -07:00
A.J. Beamon
fee131f24d
Merge pull request #7591 from sfc-gh-ajbeamon/key-backed-types-range-read-more
...
Add support for returning a more flag from key backed types' range reads
2022-07-20 15:57:50 -07:00
A.J. Beamon
190ad8c7e9
Convert existing tuple usages to use Tuple::makeTuple()
2022-07-19 13:45:59 -07:00
Markus Pilman
1de37afd52
Make TEST macros C++ only ( #7558 )
...
* proof of concept
* use code-probe instead of test
* code probe working on gcc
* code probe implemented
* renamed TestProbe to CodeProbe
* fixed refactoring typo
* support filtered output
* print probes at end of simulation
* fix missed probes print
* fix deduplication
* Fix refactoring issues
* revert bad refactor
* make sure file paths are relative
* fix more wrong refactor changes
2022-07-19 13:15:51 -07:00
A.J. Beamon
1b81e72604
Add a Tuple::makeTuple function to easily construct a tuple. Update Tuple to allow all types to be passed via .append() so they can be used with makeTuple.
2022-07-19 11:50:58 -07:00
A.J. Beamon
91949439d2
Add support for returning whether a key-backed type range read has more results or not.
2022-07-13 15:46:10 -07:00
A.J. Beamon
62683b4140
Update key backed types to support a templated Codec. This required updating the existing Codec to convert from objects into byte strings through tuples.
2022-07-01 10:11:02 -07:00
A.J. Beamon
1dc78a3b3f
Add support for different transaction types (Reference<ITransaction>, Reference<ReadYourWritesTransaction>) to key backed types
2022-07-01 10:02:39 -07:00
Markus Pilman
ffaf15c12a
moved wellknownendpoints and fixed some includes
2022-06-23 17:03:53 -06:00
Markus Pilman
d35445a868
enforce include modularization in cmake
2022-06-23 14:37:35 -06:00
Renxuan Wang
0a332ee1c1
Add proxy option to backup and restore params.
2022-03-28 17:10:49 -07:00
sfc-gh-tclinkenbeard
a71099471b
Update copyright header dates
2022-03-21 13:36:23 -07:00
Jingyu Zhou
1a5bf25b5c
Update code base to use fmt 8.1.1
2022-03-04 15:52:06 -08:00
Renxuan Wang
f7eb66441d
Try eliminating warnings in macOS and Windows CI builds.
...
MacOS warnings are format warnings, e.g., `format specifies type 'long' but the argument has type 'Version' (aka 'long long')`.
Windows warnings are `ACTOR does not contain a wait() statement`.
2022-02-25 19:06:57 -08:00
A.J. Beamon
250a88e682
Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement.
2022-02-24 12:25:52 -08:00
sfc-gh-tclinkenbeard
ec64890ac1
Remove some usages of PRId64 by using fmt library
2021-11-30 23:35:36 -08:00
sfc-gh-tclinkenbeard
ebcc023b6f
Enable missing-field-initializers clang warning
2021-11-01 14:18:31 -07:00
Chaoguang Lin
65956ae6b7
Refactor configure command; refactor changeConfig to template code to reuse existing tests
2021-09-21 10:06:04 -07:00
FDB Formatster
2c788c233d
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-08-27 17:07:47 -07:00
Jingyu Zhou
426e906a87
Remove duplicated code
2021-08-01 22:39:28 -07:00
Steve Atherton
507c1f11e3
Add .log() to bare TraceEvent() invocations without any .detail()s to avoid clang-tidy warning about immediate destruction of object without use.
2021-07-26 19:55:10 -07:00
Steve Atherton
f596a81073
Rename ::TRUE and ::FALSE in BooleanParams to ::True and ::False so as to not conflict with the TRUE and FALSE macros provided by the Windows and MacOS SDKs.
2021-07-17 00:11:40 -07:00
sfc-gh-tclinkenbeard
382533c4e1
Add FDB_ prefix to BOOLEAN_PARAM macros
2021-07-09 05:42:14 -07:00
sfc-gh-tclinkenbeard
020371a78f
Merge remote-tracking branch 'origin/master' into add-boolean-param
2021-07-07 16:50:51 -07:00
Trevor Clinkenbeard
f5ade03538
Merge pull request #4233 from sfc-gh-tclinkenbeard/encrypt-backup-files
...
Added AsyncFileEncrypted
2021-07-07 13:28:28 -07:00
sfc-gh-tclinkenbeard
8b2d8448d7
Move BackupAgentBase method implementations to .cpp file
2021-07-04 19:09:03 -07:00
sfc-gh-tclinkenbeard
8370d92c20
Expand use of BOOLEAN_PARAM in backup code
2021-07-04 18:30:55 -07:00
sfc-gh-tclinkenbeard
cadfdc551e
Use BOOLEAN_PARAM for backup agents
2021-07-04 16:27:26 -07:00
sfc-gh-tclinkenbeard
49c680b252
Use BOOLEAN_PARAM for TaskBucket
2021-07-04 13:14:25 -07:00
sfc-gh-tclinkenbeard
8cc40e3a2b
Expand use of BOOLEAN_PARAM
2021-07-02 21:41:50 -07:00
sfc-gh-tclinkenbeard
f5aa3df917
Add --encryption_key_file command line argument to fdbbackup and fdbrestore
2021-06-25 15:11:03 -07:00
sfc-gh-tclinkenbeard
53f5cd2453
Support passing encryption file to BackupContainer::openContainer
2021-06-25 14:11:21 -07:00
