266 Commits

Author SHA1 Message Date
A.J. Beamon
57390bfbc5
Merge pull request #3263 from apple/release-6.2
Merge release-6.2 into release-6.3
2020-05-28 14:14:38 -07:00
A.J. Beamon
382666d21e Permissions were wrong on installed files. A missing DESTINATION for fdbserver caused the binary to be missing. 2020-05-28 10:33:09 -07:00
A.J. Beamon
b3c269351a
Merge pull request #3252 from apple/release-6.2
Merge release-6.2 into release-6.3
2020-05-27 19:13:24 -07:00
A.J. Beamon
96f396ec7f Make sure to build the binary stripping targets when packaging stripped binaries 2020-05-27 16:40:07 -07:00
A.J. Beamon
d6be580319 Merge branch 'release-6.2' into merge-release-6.2-into-release-6.3
# Conflicts:
#	documentation/sphinx/source/command-line-interface.rst
#	documentation/sphinx/source/release-notes.rst
#	fdbcli/fdbcli.actor.cpp
2020-05-27 11:42:27 -07:00
A.J. Beamon
e4b1489a0e
Merge pull request #3202 from ajbeamon/cmake-use-stripped-binaries-for-packages
cmake: use stripped binaries for packages unless generating debug packages
2020-05-27 11:36:38 -07:00
A.J. Beamon
2f9cd383ea Use stripped executables in packages when not building with debug symbols. 2020-05-21 13:39:55 -07:00
Evan Tschannen
06676fc88f
Merge pull request #3166 from tclinken/6.3-avoid-ub
Eliminate some undefined behavior and const_cast usage
2020-05-20 14:36:30 -07:00
Evan Tschannen
3b9e9a9257
Merge pull request #3210 from ajbeamon/remove-include-version-h
Remove IncludeVersion.h
2020-05-20 14:12:23 -07:00
A.J. Beamon
4b5e94cf27
Merge pull request #3208 from apple/release-6.2
Merge release-6.2 into release-6.3
2020-05-20 13:47:48 -07:00
A.J. Beamon
b589c3c940 Remove IncludeVersions.h, which is no longer needed for cmake-only builds 2020-05-20 13:23:02 -07:00
A.J. Beamon
54c97fe2bc Backup's ProgramStart command line string would be incomplete if arguments were specified using '=' 2020-05-20 09:38:33 -07:00
A.J. Beamon
59e33968c6 Remove erroneous restore options. 2020-05-20 08:57:56 -07:00
tclinken
667b224fc8 Several changes to reduce unnecessary moves/copies 2020-05-13 19:59:34 -07:00
Evan Tschannen
f17f00fdd5 Merge branch 'release-6.2'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
2020-05-10 22:33:38 -07:00
Meng Xu
c237ca9708
Merge pull request #3081 from jzhou77/backup-cmd
Update fdbbackup documentation on new "-p" option
2020-05-04 14:17:34 -07:00
Jingyu Zhou
2c5da1c116 Remove "-p" option and use "--partitioned_log_experimental" option 2020-05-04 13:45:54 -07:00
Alex Miller
78d644f673
Merge branch 'master' into bugfixes/cmake-includes 2020-05-04 02:11:46 -07:00
Meng Xu
37a537c2a3 Fix MacOS compilation error 2020-04-30 18:30:15 -07:00
Steve Atherton
02aeab1edd Re-add improperly named options in order to not break compatibility with anything depending on them in a patch release. 2020-04-30 16:23:30 -07:00
Meng Xu
07a9a05683 FastRestore:Agent:Fix restore requests 2020-04-30 16:20:20 -07:00
Steve Atherton
697beb3ebb Fix fdbrestore parameters with one leading hyphen instead of two, which did not match documentation. 2020-04-30 12:13:35 -07:00
Meng Xu
3f510d0653
Merge pull request #3036 from jzhou77/backup-cmd
Several bug fixes for new backups
2020-04-29 16:34:59 -07:00
Meng Xu
e09394459a Merge branch 'master' into mengxu/fr-circus 2020-04-28 22:08:29 -07:00
Meng Xu
8742cc0ab6 FastRestore:Fix windows build 2020-04-28 19:12:27 -07:00
Jingyu Zhou
7d59e53349 Consolidate makePadding() 2020-04-28 15:39:23 -07:00
Meng Xu
f5e8345496 FastRestoreAgent:Use atomicParallelRestore to kick off restore
Replace the handcrafted version with atomicParallelRestore actor
which is simulation tested
2020-04-27 22:15:00 -07:00
Meng Xu
d4509090d4 Generate fastrestore_agent symlink in CMake 2020-04-23 07:55:44 -07:00
A.J. Beamon
af4e0088ba
Merge pull request #2896 from tclinken/atomically-update-dependent-knobs
Atomically update dependent knobs
2020-04-08 15:00:49 -07:00
tclinken
52860043c9 Merge remote-tracking branch 'origin' into atomically-update-dependent-knobs 2020-04-08 12:26:21 -07:00
Markus Pilman
7ba2cec9ef Merge branch 'master' of github.com:apple/foundationdb into features/no-make 2020-04-08 10:23:46 -07:00
Markus Pilman
d4542dbb5a Delete old build system 2020-04-07 11:03:45 -07:00
Jingyu Zhou
9fb3fb9d82 Add pause/resume for new backups
To pause/resume the backup workers, the fdbbackup command will write to the
backupPausedKey. Then backup workers noticed the value of the key has been
changed and stops/resumes pulling from TLog.
2020-04-06 14:29:46 -07:00
Markus Pilman
e4611e8ae4 fix versions.h stupidity 2020-04-06 10:28:55 -07:00
Markus Pilman
8b5780c36c don't include source and binary dir
This forces users to use include paths from the sources root.

So `#include "Arena.h"` won't work anymore, only
`#include "flow/Arena.h"` will.
2020-04-06 10:13:49 -07:00
Markus Pilman
bbd2fe62cc Merge branch 'master' of github.com:apple/foundationdb into features/boost70 2020-04-02 09:21:01 -07:00
Meng Xu
4cf796e838
Merge pull request #2891 from jzhou77/backup-cmd
Add an option for fdbbackup to use new backup system
2020-04-01 16:12:10 -07:00
tclinken
884e92bb49 Atomically update dependent knobs 2020-04-01 15:18:49 -07:00
Jingyu Zhou
906174e3e8 Add an option for fdbbackup to use new backup system
I.e., "-p", or "--partitioned_log" to enable it. By default, old backup system
is used.
2020-03-31 14:54:32 -07:00
Jingyu Zhou
3c76722504 Fix valgrind error of invalid memory access 2020-03-30 16:38:22 -07:00
Jingyu Zhou
3c32835cce Fix decoder for unfinished version batch in a log
A mutation log's version batch data can be split into multiple blocks, and some
of the blocks can be spread across two mutation logs. Thus, the decoder needs
copy unfinished version batch data from previous file progress to the next one.
2020-03-30 11:34:51 -07:00
Markus Pilman
28cd38913d Merge branch 'master' of github.com:apple/foundationdb into features/boost70 2020-03-27 13:44:10 -07:00
Jingyu Zhou
feedab02a0
Merge pull request #2855 from xumengpanda/mengxu/fr-api-atomicrestore-PR
Add ApiCorrectnessAtomicRestore workload for the new performant restore
2020-03-25 18:05:26 -07:00
Meng Xu
a93f13cfd7 Remove redundant restoreRequestDone break in backup.actor 2020-03-25 15:19:46 -07:00
Meng Xu
495afe2e0b Improve how to wati for restore to finish
Remove default parameter for atomicRestore as suggested in review.
2020-03-25 13:54:21 -07:00
Jingyu Zhou
7831bec2b0 Address review comments on trace events 2020-03-24 10:54:12 -07:00
Jingyu Zhou
b18f192831 Fix decode bug of missing mutations
After reading a new block, all mutations are sorted by version again, which
can invalidate previously tuple. As a result, the decoded file will miss some
of the mutations.
2020-03-20 20:15:09 -07:00
Jingyu Zhou
ca1a4ef9fd Ignore mutation logs of size 0 in converter 2020-03-20 20:15:08 -07:00
Jingyu Zhou
d0a24dd20d Decode out of order mutations in old mutation logs
In the old mutation logs, a version's mutations are serialized as a buffer.
Then the buffer is split into smaller chunks, e.g., 10000 bytes each. When
writting chunks to the final mutation log file, these chunks can be flushed
out of order. For instance, the (version, chunck_part) can be in the order of
(3, 0), (4, 0), (3, 1). As a result, the decoder must read forward to find all
chunks of data for a version.

Another complication is that the files are organized into blocks, where (3, 1)
can be in a subsequent block. This change checks the value size for each
version, if the size is smaller than the right size, the decoder will look
for the missing chucks in the next block.
2020-03-20 20:15:08 -07:00
Jingyu Zhou
20df67ee6a Filter partitioned logs with subset relationship
If a log file's progress is not saved, a new log file will be generated
with the same begin version. Then we can have a file that contains a subset
of contents in another log file. During restore, we should filter out files
that their contents are subset of other files.
2020-03-20 20:15:08 -07:00