334 Commits

Author SHA1 Message Date
Alvin Moore
7628d04fb9 Merge branch 'release-6.2' of github.com:apple/foundationdb into release_6.2_merge
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
2020-01-09 07:21:16 -08:00
Steve Atherton
4ff058e86b Backup and DR layer status document generation now uses snapshot reads for all keys read to avoid unnecessary conflicts when read during a status update or cleanup transaction. Since many of the keys read use wrapper functions, all of the underlying functions in BackupAgentBase and its two implementations also required a snapshot mode argument. All snapshot arguments default to false to match the underlying FDB API get/getrange methods. 2019-12-19 00:29:35 -08:00
Alvin Moore
3bf971ba8b Merge branch 'release-6.2' of github.com:apple/foundationdb into release_6.2_merge
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbserver/storageserver.actor.cpp
2019-12-12 07:13:12 -08:00
Alvin Moore
363feafacd Changed the name of a cmake functions
Added working directory for cmake command
2019-12-11 13:59:03 -08:00
Alvin Moore
aece864143 Added support for creating symlinks to fdbbackup within bin and packages/bin directory 2019-12-10 11:44:45 -08:00
A.J. Beamon
ed8d3f163c Rename hgVersion to sourceVersion. 2019-11-15 12:26:51 -08:00
Evan Tschannen
4de60fc437 Merge branch 'release-6.2'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbserver/TLogServer.actor.cpp
2019-11-01 15:48:04 -07:00
A.J. Beamon
f175ed30b3 Cleanup the fdbbackup cleanup command output. Add cleanup to the usage output printed for fdbbackup. 2019-10-31 09:52:21 -07:00
Andrew Noyes
d4de608bb6 Fix OPEN_FOR_IDE build 2019-10-25 10:42:22 -07:00
Meng Xu
7599dd0093
Merge pull request #2205 from jzhou77/backup-fix
Fix format string with more portable code
2019-10-02 20:03:35 -07:00
Jingyu Zhou
ceb39c0279 Fix format string with more portable code 2019-10-02 15:25:14 -07:00
Meng Xu
d0147e5e5d Merge branch 'release-6.2' into mengxu/merge-release620-to-master-v3
Resolved Conflicts:
	documentation/sphinx/source/release-notes.rst
	fdbserver/DataDistribution.actor.cpp
	versions.target
2019-10-02 13:22:56 -07:00
Evan Tschannen
9463b91594
Update fdbbackup/backup.actor.cpp
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-09-30 13:16:55 -07:00
Evan Tschannen
4d659662b8 made cleanup handle retries better 2019-09-30 12:44:20 -07:00
Evan Tschannen
ef01ad2ed8 optimized log range clearing to clear everything for each possible hash (256 clears) if that would be more efficient than one clear per second that has elapsed
aborting a DR without the —cleanup flag will still attempt to cleanup for 30 seconds before giving up
added a cleanup command to fdbbackup which can remove mutations from orphaned DRs which were stopped without the —cleanup flag
2019-09-27 18:32:27 -07:00
Alex Miller
9a64dc7f24
Merge pull request #1792 from ajbeamon/remove-dl-iterate-phdr-hack
Remove dl_iterate_phdr caching used by slow task profiler
2019-09-17 16:51:30 -07:00
Evan Tschannen
b495cc697b Merge branch 'release-6.2'
# Conflicts:
#	CMakeLists.txt
#	documentation/sphinx/source/release-notes.rst
#	versions.target
2019-09-13 09:25:08 -07:00
Andrew Noyes
bd7678e71b Remove --object-serializer help text 2019-09-05 11:57:59 -07:00
Meng Xu
c2355f721e Merge branch 'master' into mengxu/performant-restore-PR 2019-09-04 17:11:42 -07:00
Meng Xu
d160810662 FastRestore:Resolve review comments 2019-09-04 16:48:43 -07:00
A.J. Beamon
e465cbef07 Merge branch 'master' into remove-dl-iterate-phdr-hack
# Conflicts:
#	fdbserver/fdbserver.actor.cpp
2019-08-22 10:13:23 -07:00
A.J. Beamon
f02799455e Add --loggroup to fdbserver and fdbbackup help text. 2019-08-19 12:59:14 -07:00
Meng Xu
7ff46e6772 Merge branch 'master' into mengxu/performant-restore-PR 2019-08-07 20:31:56 -07:00
mpilman
370ba8b841 Remove --object-serializer flag from executables 2019-08-06 09:25:40 -07:00
Meng Xu
3b54363780 FastRestore:Apply Clang-format 2019-08-01 18:09:12 -07:00
Meng Xu
7ccaeddf05 Merge branch 'master' into mengxu/performant-restore-PR 2019-08-01 13:23:17 -07:00
A.J. Beamon
b5d2234a13 Merge branch 'release-6.1' into merge-release-6.1-into-master
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbbackup/backup.actor.cpp
#	fdbserver/MoveKeys.actor.cpp
#	flow/FastAlloc.h
#	versions.target
2019-07-30 16:23:42 -07:00
Stephen Atherton
9e1042d903 Bug fix, restore with --dryrun option would still require the --dest_cluster_file option even though it does not connect to a cluster. 2019-07-29 16:21:12 -07:00
Stephen Atherton
b644f15b87 Bug fix: fdbrestore commands other than "start" were using default cluster file argument handling (but without the -C flag) instead of using the --dest_cluster_file argument. 2019-07-29 13:19:28 -07:00
Meng Xu
b0c31f28af FastRestore:Fix bug that blocks restore
1) Should recruit only configured number of roles;
2) Should never register a restore master interface as a restore worker (loader or applier) interface.
2019-07-25 17:55:37 -07:00
Meng Xu
45083edf74 Merge branch 'master' into mengxu/performant-restore-PR
Fix conflicts as well.
2019-07-25 10:46:11 -07:00
mpilman
1ac2d01b03 Merge remote-tracking branch 'upstream/master' into flatbuffers-fixes2 2019-07-18 09:50:08 -07:00
mpilman
d5caf0c1b4 Merge branch 'flatbuffers-fixes2' of github.com:mpilman/foundationdb into flatbuffers-fixes2 2019-07-16 14:47:40 -07:00
A.J. Beamon
d5051b08dd Make trace event field lengths (and total event sizes) default knobified and configurable. Add a transaction option to control the field length of transaction debug logging. Make the program start command line field less likely to be truncated. 2019-07-12 16:12:35 -07:00
Andrew Noyes
5be0c47409 Add some missing help text 2019-07-12 10:46:46 -07:00
Andrew Noyes
269f077196 Add --object-serializer flag to fdbbackup etc 2019-07-12 10:46:45 -07:00
A.J. Beamon
a5a6f8431c Add a random UID to TransactionMetrics in case a client opens multiple connections and also a field to indicate whether the connection is internal. Convert some of the metrics to our Counter object instead of running totals. 2019-07-08 14:01:04 -07:00
A.J. Beamon
6a899ddff3 Remove dl_iterate_phdr results caching that is used by slow task profiler, instead favoring disabling and reenabling profiling around the call. Add a mechanism to handle deferred profile requests. 2019-07-03 12:48:36 -07:00
Evan Tschannen
37c1df2491
Merge pull request #1705 from bnamasivayam/suspend-process
Extend RebootRequest API to include time to suspend the process befor…
2019-06-20 17:36:25 -07:00
mpilman
68ce9a5e75 ProtocolVersion type - second try 2019-06-18 17:55:27 -07:00
Evan Tschannen
20e3edeb0a Merge branch 'release-6.1'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbserver/storageserver.actor.cpp
#	versions.target
2019-06-14 12:42:59 -07:00
Balachandar Namasivayam
5eb833759e Extend RebootRequest API to include time to suspend the process before reboot. This is intended to be used for testing purposes to simulate failures. 2019-06-14 11:35:38 -07:00
Stephen Atherton
235d24ee26 Bug fix: fdbrestore abort, wait, and status were not using the --dest_cluster_file argument. 2019-06-13 17:42:11 -07:00
Meng Xu
3fcb6ec0a1 FastRestore:Refactor RestoreLoader and fix bugs
Refactor RestoreLoader code and
Fix a bug in notifying restore finish.
2019-06-04 21:53:31 -07:00
Meng Xu
477fd152c0 FastRestore:Refactor code
1) Use the runRYWTransaction for simple DB access
2) Replace some printf with TraceEvent
3) Remove printf not used in debugging
4) Avoid wait inside the condition in loop-choose-when for
   the core routine of restore worker, loader and applier.
5) Rename Restore.actor.cpp to RestoreWorker.actor.cpp since
   the file only has functionalities related to restore worker.

Passed correctness test
2019-06-04 11:22:47 -07:00
Meng Xu
a3f61e6df7 FastRestore:Rafctor:Reduce code size
1) Use runRYWTransaction to replace the loop-try style;
2) Remove unnecessary printf
3) Do not mistakenly send reply twice.
2019-05-29 17:03:50 -07:00
A.J. Beamon
603721e125 Merge branch 'master' into thread-safe-random-number-generation
# Conflicts:
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/genericactors.actor.cpp
#	fdbrpc/sim2.actor.cpp
#	fdbserver/DiskQueue.actor.cpp
#	fdbserver/workloads/BulkSetup.actor.h
#	flow/ActorCollection.actor.cpp
#	flow/Net2.actor.cpp
#	flow/Trace.cpp
#	flow/flow.cpp
2019-05-23 08:35:47 -07:00
Alvin Moore
3acaa7343e Enabled C++17 for all Windows projects
Set Visual Studio version to 2017 (first version to support C++17)
2019-05-16 17:44:13 -07:00
Alvin Moore
94aed513c7 Switched Windows tools within projects to 2017 2019-05-16 15:05:11 -07:00
A.J. Beamon
5f55f3f613 Replace g_random and g_nondeterministic_random with functions deterministicRandom() and nondeterministicRandom() that return thread_local random number generators. Delete g_debug_random and trace_random. Allow only deterministicRandom() to be seeded, and require it to be seeded from each thread on which it is used. 2019-05-10 14:01:52 -07:00