317 Commits

Author SHA1 Message Date
A.J. Beamon
7bca503528 Use narrow includes in key backed types; remove some unnecessary return statements. 2022-07-14 13:26:43 -07:00
A.J. Beamon
1dc78a3b3f Add support for different transaction types (Reference<ITransaction>, Reference<ReadYourWritesTransaction>) to key backed types 2022-07-01 10:02:39 -07:00
Markus Pilman
03d913a1de Flow compiling 2022-06-27 17:05:55 -06:00
Markus Pilman
d35445a868 enforce include modularization in cmake 2022-06-23 14:37:35 -06:00
Renxuan Wang
13b2b455b1 HTTP proxy for backup.
Only support non-TLS blobs now.
2022-05-09 13:56:33 -07:00
sfc-gh-tclinkenbeard
06825775db Fix formatting of lines with TLS_OPTION_FLAGS 2022-05-02 22:56:06 -07:00
sfc-gh-tclinkenbeard
7f05221cfe Removed TLS_DISABLED macro 2022-05-02 22:15:27 -07:00
Yi Wu
994b8c92f8
Add option to limit resident memory and remove default memory limit (#6719)
Changing `memory` option to limit resident memory instead of virtual memory, in config file and fdbserver/fdbbackup/fdbcli command-line argument. Since `rlimit` doesn't support limiting virtual memory, the current implementation have both of fdbmonitor and the fdbserver/fdbbackup process checking process RSS periodically and kill and restart the process if the limit is exceeded.

Adding a new `memory_vsize` option to limit virtual memory, if backward-compatible behavior is desired.

closes #6671, closes #6672
2022-04-06 20:06:24 -07:00
Jingyu Zhou
f68fd28d73 Refactor duplicated code into IKnobCollection::setupKnobs() 2022-04-05 02:06:38 -07:00
Renxuan Wang
e1775627ab Add a check on proxy format. 2022-03-29 10:17:14 -07:00
Renxuan Wang
0a332ee1c1 Add proxy option to backup and restore params. 2022-03-28 17:10:49 -07:00
sfc-gh-tclinkenbeard
a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
sfc-gh-tclinkenbeard
8dcac2f76d Fix typos 2022-03-13 10:02:11 -03:00
Jingyu Zhou
1a5bf25b5c Update code base to use fmt 8.1.1 2022-03-04 15:52:06 -08:00
Renxuan Wang
233c918ffb Replace printf() and fprintf() with fmt::print(). 2022-02-25 19:06:57 -08:00
Renxuan Wang
f7eb66441d Try eliminating warnings in macOS and Windows CI builds.
MacOS warnings are format warnings, e.g., `format specifies type 'long' but the argument has type 'Version' (aka 'long long')`.
Windows warnings are `ACTOR does not contain a wait() statement`.
2022-02-25 19:06:57 -08:00
A.J. Beamon
250a88e682 Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement. 2022-02-24 12:25:52 -08:00
sfc-gh-tclinkenbeard
ef711d405c Add fdbbackup tags command 2022-01-24 13:34:54 -08:00
A.J. Beamon
ff1cb58174 Convert hyphens to underscores for all prefix-based arguments (e.g. --knob-, --locality-) 2021-12-14 12:01:44 -08:00
A.J. Beamon
f24adc7b6a Fix a bunch of places where we used old-style arguments. Allow hyphens for profiler args. 2021-12-14 09:59:14 -08:00
A.J. Beamon
f29f487823
Unify flags (#25)
* Unify flags implementation and change help text in backup.actor.cpp
Description

Testing

* Keep LOG_GROUP unchanged

Description

Testing

* Transfer the hyphens to underscores for internal options and user's input, EXCEPT leading hyphens

Description

Testing

* Use a deep copy of the user's input flag to do the match

Description

Testing

* Convert the _ to - in Option arrays of backup.actor.cpp

Description

Testing

* Transter _ to - for files:
        TLSConfig.actor.h, fdbcli.actor.cpp, fdbserver.actor.cpp, FileConverter.h, FileConverter.cpp

Description

Testing

* Change another way to unify flag: using SO_O_ICASE_HYPHEN_AND_UNDERSCORE to determine whether we do the conversion in function IsEqual

Description

Testing

* Change the config command's name from SO_O_ICASE_HYPHEN_AND_UNDERSCORE to SO_O_HYPHEN_TO_UNDERSCORE

Description

Testing

* Update the comment for the SO_O_HYPHEN_TO_UNDERSCORE

Description

Testing

* Fix left underscore in SOption arrays

Description

Testing

* Convert _ to - in several files for commands

Description

Testing

* Make the FDBService and fdbmonitor backward compatible

Description

Testing

* Fix bugs about pointers

Description

Testing

* Check underscore and hyphen at the same time for --knob_, --localily_ and --test_
And fix bugs in fdbmonitor and FDBService
Description

Testing

* Simplify the function in fdbmonitor and FDBService about retrieving arguments.
And fix some documents in masterserver.actor.cpp

Description

Testing

* Convert _ to - for knob in the setKnob functions

Description

Testing

* Convert - to _ in the setKnob functions

Description
Since key in the knob related maps only contain _

Testing

* Rename varialbe name in the fdbmonitor and FDBService for clarification

Description

Testing

Co-authored-by: Chang Liu <chang.liu@snowflake.com>
2021-12-14 08:44:39 -08:00
sfc-gh-tclinkenbeard
464d9488ef Merge remote-tracking branch 'origin/master' into fix-unused-warnings 2021-12-01 23:52:09 -08:00
sfc-gh-tclinkenbeard
90ced244eb Fix -Wunused-but-set-variable warnings 2021-12-01 18:15:53 -08:00
sfc-gh-tclinkenbeard
ec64890ac1 Remove some usages of PRId64 by using fmt library 2021-11-30 23:35:36 -08:00
sfc-gh-tclinkenbeard
deb0eb85e8 Initialize global flow knobs in setupNetwork 2021-11-16 00:41:34 -08:00
A.J. Beamon
e882eb33fc Abstract the cluster file into a cluster connection record that can be backed by something other than the filesystem. 2021-10-22 11:05:18 -07:00
Xiaoge Su
e68b131e4a fixup! Reformat source code 2021-09-16 19:40:28 -07:00
Xiaoge Su
abf73047ca Enforce std:: specifier rather than using namespace 2021-09-16 19:40:28 -07:00
A.J. Beamon
d4c67eb13d Fix crash when DR binaries are invoked without a source cluster file. Update the usage text for dr_agent cluster files to include the long form options. 2021-08-20 12:55:13 -07:00
Vishesh Yadav
de93e2f854
Update fdbbackup/backup.actor.cpp
Co-authored-by: Jingyu Zhou <jingyuzhou@gmail.com>
2021-08-04 20:27:43 -07:00
Vishesh Yadav
98fa70da11 backup: Show error info for RemovedDeadBackupLayerStatus 2021-08-04 13:28:47 -07:00
Steve Atherton
f596a81073 Rename ::TRUE and ::FALSE in BooleanParams to ::True and ::False so as to not conflict with the TRUE and FALSE macros provided by the Windows and MacOS SDKs. 2021-07-17 00:11:40 -07:00
sfc-gh-tclinkenbeard
020371a78f Merge remote-tracking branch 'origin/master' into add-boolean-param 2021-07-07 16:50:51 -07:00
Trevor Clinkenbeard
f24581b644
Merge pull request #5120 from sfc-gh-tclinkenbeard/improve-poll-delay-safety
Improve safety of backup agent poll delay
2021-07-07 14:42:33 -07:00
sfc-gh-tclinkenbeard
3d0536dcec Improve safety of backup agent poll delay 2021-07-05 13:22:31 -07:00
sfc-gh-tclinkenbeard
8370d92c20 Expand use of BOOLEAN_PARAM in backup code 2021-07-04 18:30:55 -07:00
sfc-gh-tclinkenbeard
cadfdc551e Use BOOLEAN_PARAM for backup agents 2021-07-04 16:27:26 -07:00
sfc-gh-tclinkenbeard
8cc40e3a2b Expand use of BOOLEAN_PARAM 2021-07-02 21:41:50 -07:00
sfc-gh-tclinkenbeard
79ff07a071 Added *BOOLEAN_PARAM macros to enforce documentation of boolean parameters 2021-07-02 15:04:42 -07:00
sfc-gh-tclinkenbeard
27e44c1bb9 Add support for encryption testing in BackupCorrectness workload 2021-06-26 11:15:12 -07:00
sfc-gh-tclinkenbeard
1afae7623b Added /backup/containers/localdir/encrypted unit test 2021-06-25 22:33:26 -07:00
sfc-gh-tclinkenbeard
f5aa3df917 Add --encryption_key_file command line argument to fdbbackup and fdbrestore 2021-06-25 15:11:03 -07:00
sfc-gh-tclinkenbeard
41c790b299 Merge remote-tracking branch 'origin/master' into config-db 2021-06-10 22:31:23 -07:00
sfc-gh-tclinkenbeard
c272304e60 Manage global flow knobs with global knob collection 2021-06-09 22:33:00 -07:00
sfc-gh-tclinkenbeard
83a0e473e8 Refactor IKnobCollection code 2021-06-09 20:50:00 -07:00
sfc-gh-tclinkenbeard
89dadf1475 Create IKnobCollection interface 2021-06-02 23:40:52 -07:00
sfc-gh-tclinkenbeard
4c059670a0 Parse configuration knob values on the client 2021-06-02 10:04:46 -07:00
sfc-gh-tclinkenbeard
f28ac955c3 Remove unnecessary temporary objects while growing objects of type std::vector<std::pair<A, B>> 2021-05-10 16:32:50 -07:00
sfc-gh-tclinkenbeard
5c2d7b6080 Create RangeResult type alias 2021-05-03 13:14:16 -07:00
RenxuanW
719f810676 Rename incrementalBackupOnly to onlyAppyMutationLogs in all restore configs and functions. 2021-04-26 12:30:46 -07:00