129 Commits

Author SHA1 Message Date
FDB Formatster
0c214d71da apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-05 16:31:35 -06:00
A.J. Beamon
54c97fe2bc Backup's ProgramStart command line string would be incomplete if arguments were specified using '=' 2020-05-20 09:38:33 -07:00
Steve Atherton
02aeab1edd Re-add improperly named options in order to not break compatibility with anything depending on them in a patch release. 2020-04-30 16:23:30 -07:00
Steve Atherton
697beb3ebb Fix fdbrestore parameters with one leading hyphen instead of two, which did not match documentation. 2020-04-30 12:13:35 -07:00
Alex Miller
595dd77ed1 Merge remote-tracking branch 'upstream/release-6.2' into certificate-refresh 2020-03-04 20:25:42 -08:00
Alex Miller
9b5ef3416e Refactor TLSParams into TLSConfig + LoadedTLSConfig
The idea being that we keep around a TLSConfig that the configuration
that the user has provided, and then when we want to intialize an SSL
context, we ask the TLSConfig to load all certificates and return us a
LoadedTLSConfig that is a concrete set of certificate bytes in memory.

initTLS now just takes the in-memory bytes and applies them to the ssl
context.

This is a large refactor to lead up into certificate refeshing, where we
will periodically check for changes to the certificates, and then
re-load them and apply them to a new SSL context.
2020-03-04 20:14:47 -08:00
Evan Tschannen
cdcb816866
Update fdbbackup/backup.actor.cpp 2020-03-04 16:08:45 -08:00
A.J. Beamon
58e621eca1 Invalid knobs or knob values are treated as warnings rather than errors. Apply this change to backup as well. 2020-03-04 15:50:04 -08:00
Alex Miller
927cff3317 Report errors on TLS misconfigurations ... or at least try to. 2020-02-20 16:57:29 -08:00
Evan Tschannen
761da5a059 code cleanup 2020-02-19 17:59:45 -08:00
Steve Atherton
4ff058e86b Backup and DR layer status document generation now uses snapshot reads for all keys read to avoid unnecessary conflicts when read during a status update or cleanup transaction. Since many of the keys read use wrapper functions, all of the underlying functions in BackupAgentBase and its two implementations also required a snapshot mode argument. All snapshot arguments default to false to match the underlying FDB API get/getrange methods. 2019-12-19 00:29:35 -08:00
A.J. Beamon
f175ed30b3 Cleanup the fdbbackup cleanup command output. Add cleanup to the usage output printed for fdbbackup. 2019-10-31 09:52:21 -07:00
Evan Tschannen
9463b91594
Update fdbbackup/backup.actor.cpp
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-09-30 13:16:55 -07:00
Evan Tschannen
4d659662b8 made cleanup handle retries better 2019-09-30 12:44:20 -07:00
Evan Tschannen
ef01ad2ed8 optimized log range clearing to clear everything for each possible hash (256 clears) if that would be more efficient than one clear per second that has elapsed
aborting a DR without the —cleanup flag will still attempt to cleanup for 30 seconds before giving up
added a cleanup command to fdbbackup which can remove mutations from orphaned DRs which were stopped without the —cleanup flag
2019-09-27 18:32:27 -07:00
Andrew Noyes
bd7678e71b Remove --object-serializer help text 2019-09-05 11:57:59 -07:00
A.J. Beamon
f02799455e Add --loggroup to fdbserver and fdbbackup help text. 2019-08-19 12:59:14 -07:00
mpilman
370ba8b841 Remove --object-serializer flag from executables 2019-08-06 09:25:40 -07:00
A.J. Beamon
b5d2234a13 Merge branch 'release-6.1' into merge-release-6.1-into-master
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbbackup/backup.actor.cpp
#	fdbserver/MoveKeys.actor.cpp
#	flow/FastAlloc.h
#	versions.target
2019-07-30 16:23:42 -07:00
Stephen Atherton
9e1042d903 Bug fix, restore with --dryrun option would still require the --dest_cluster_file option even though it does not connect to a cluster. 2019-07-29 16:21:12 -07:00
Stephen Atherton
b644f15b87 Bug fix: fdbrestore commands other than "start" were using default cluster file argument handling (but without the -C flag) instead of using the --dest_cluster_file argument. 2019-07-29 13:19:28 -07:00
mpilman
1ac2d01b03 Merge remote-tracking branch 'upstream/master' into flatbuffers-fixes2 2019-07-18 09:50:08 -07:00
mpilman
d5caf0c1b4 Merge branch 'flatbuffers-fixes2' of github.com:mpilman/foundationdb into flatbuffers-fixes2 2019-07-16 14:47:40 -07:00
A.J. Beamon
d5051b08dd Make trace event field lengths (and total event sizes) default knobified and configurable. Add a transaction option to control the field length of transaction debug logging. Make the program start command line field less likely to be truncated. 2019-07-12 16:12:35 -07:00
Andrew Noyes
5be0c47409 Add some missing help text 2019-07-12 10:46:46 -07:00
Andrew Noyes
269f077196 Add --object-serializer flag to fdbbackup etc 2019-07-12 10:46:45 -07:00
A.J. Beamon
a5a6f8431c Add a random UID to TransactionMetrics in case a client opens multiple connections and also a field to indicate whether the connection is internal. Convert some of the metrics to our Counter object instead of running totals. 2019-07-08 14:01:04 -07:00
Evan Tschannen
37c1df2491
Merge pull request #1705 from bnamasivayam/suspend-process
Extend RebootRequest API to include time to suspend the process befor…
2019-06-20 17:36:25 -07:00
mpilman
68ce9a5e75 ProtocolVersion type - second try 2019-06-18 17:55:27 -07:00
Evan Tschannen
20e3edeb0a Merge branch 'release-6.1'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbserver/storageserver.actor.cpp
#	versions.target
2019-06-14 12:42:59 -07:00
Balachandar Namasivayam
5eb833759e Extend RebootRequest API to include time to suspend the process before reboot. This is intended to be used for testing purposes to simulate failures. 2019-06-14 11:35:38 -07:00
Stephen Atherton
235d24ee26 Bug fix: fdbrestore abort, wait, and status were not using the --dest_cluster_file argument. 2019-06-13 17:42:11 -07:00
A.J. Beamon
5f55f3f613 Replace g_random and g_nondeterministic_random with functions deterministicRandom() and nondeterministicRandom() that return thread_local random number generators. Delete g_debug_random and trace_random. Allow only deterministicRandom() to be seeded, and require it to be seeded from each thread on which it is used. 2019-05-10 14:01:52 -07:00
Austin Seipp
a43071339c fdbbackup: fix some print/scan format warnings
Signed-off-by: Austin Seipp <aseipp@pobox.com>
2019-05-06 13:35:29 -07:00
Andrew Noyes
ef04471a66 Fix more unused-variable warnings 2019-04-17 16:04:10 -07:00
mpilman
1c16f87a4e Remove trace-calls to printable (in non-workloads) 2019-04-05 13:12:19 -07:00
Jingyu Zhou
7c02ee6fdd Fix compiler warning about unreferenced exception variable 2019-03-26 13:43:47 -07:00
Andrew Noyes
56955deed9 Set TRACE_FORMAT network option in fdbbackup 2019-03-22 10:39:13 -07:00
Steve Atherton
8aab719c22
Merge branch 'master' into feature-backup-json 2019-03-12 18:23:16 -07:00
Stephen Atherton
bc0b2aa040 Merge branch 'release-6.0' of https://github.com/apple/foundationdb
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbbackup/backup.actor.cpp
#	fdbclient/BlobStore.actor.cpp
2019-03-12 04:49:12 -07:00
Stephen Atherton
f0eae0295f Merge branch 'master' of https://github.com/apple/foundationdb into feature-backup-json 2019-03-12 03:35:03 -07:00
Steve Atherton
041b261464
Remove short option name for active snapshot interval 2019-03-11 01:48:51 -07:00
Stephen Atherton
023bbb566f Renamed backup state enums for clarity, added backup state names. Changed Epochs to EpochSeconds in backup JSON along with some other renaming/moving of fields, and added information about snapshot dispatch. Changed timestamp format for input/output in all backup/restore contexts to be a fully qualified time with timezone offset. Added information about the last snapshot dispatch to backup config and status (not yet populated). 2019-03-10 16:00:01 -07:00
mpilman
ebffe8c633 print correct pahes in alloc instrumentation 2019-03-08 15:03:17 -08:00
Stephen Atherton
1399aee532 Added --json option to fdbbackup status. 2019-03-06 21:32:46 -08:00
Stephen Atherton
ca8bbad657 Added --json option to fdbbackup describe. Also added expired percentage indicator to snapshot details. 2019-03-06 14:14:06 -08:00
Stephen Atherton
7778112f6a Bug fix, restore was using the destination cluster to look up timestamps when printing the backup description instead of (optionally) the original cluster which generated the backup. Made missing cluster file errors more clear. 2019-03-06 02:45:55 -08:00
Stephen Atherton
888c4764d3 Bug fix: fdbrestore was not using a default range set. 2019-03-06 01:57:03 -08:00
Stephen Atherton
28e67aa041 Fixed option name to match documentation. 2019-03-05 23:26:45 -08:00
Stephen Atherton
eed8dc816d Bug fix: fdbbackup modify on a tag that has never been used before would result in an internal error. 2019-03-05 22:17:37 -08:00