52 Commits

Author SHA1 Message Date
Jingyu Zhou
cff1bc24fc Fix decoder for missing ClearRanges
If the ClearRange mutation happens within the given key prefix space,
previously the mutation is not printed. Fix by checking the prefix range
overlaps with the ClearRange mutation.
2022-03-22 16:04:56 -07:00
sfc-gh-tclinkenbeard
a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
A.J. Beamon
f24adc7b6a Fix a bunch of places where we used old-style arguments. Allow hyphens for profiler args. 2021-12-14 09:59:14 -08:00
A.J. Beamon
f29f487823
Unify flags (#25)
* Unify flags implementation and change help text in backup.actor.cpp
Description

Testing

* Keep LOG_GROUP unchanged

Description

Testing

* Transfer the hyphens to underscores for internal options and user's input, EXCEPT leading hyphens

Description

Testing

* Use a deep copy of the user's input flag to do the match

Description

Testing

* Convert the _ to - in Option arrays of backup.actor.cpp

Description

Testing

* Transter _ to - for files:
        TLSConfig.actor.h, fdbcli.actor.cpp, fdbserver.actor.cpp, FileConverter.h, FileConverter.cpp

Description

Testing

* Change another way to unify flag: using SO_O_ICASE_HYPHEN_AND_UNDERSCORE to determine whether we do the conversion in function IsEqual

Description

Testing

* Change the config command's name from SO_O_ICASE_HYPHEN_AND_UNDERSCORE to SO_O_HYPHEN_TO_UNDERSCORE

Description

Testing

* Update the comment for the SO_O_HYPHEN_TO_UNDERSCORE

Description

Testing

* Fix left underscore in SOption arrays

Description

Testing

* Convert _ to - in several files for commands

Description

Testing

* Make the FDBService and fdbmonitor backward compatible

Description

Testing

* Fix bugs about pointers

Description

Testing

* Check underscore and hyphen at the same time for --knob_, --localily_ and --test_
And fix bugs in fdbmonitor and FDBService
Description

Testing

* Simplify the function in fdbmonitor and FDBService about retrieving arguments.
And fix some documents in masterserver.actor.cpp

Description

Testing

* Convert _ to - for knob in the setKnob functions

Description

Testing

* Convert - to _ in the setKnob functions

Description
Since key in the knob related maps only contain _

Testing

* Rename varialbe name in the fdbmonitor and FDBService for clarification

Description

Testing

Co-authored-by: Chang Liu <chang.liu@snowflake.com>
2021-12-14 08:44:39 -08:00
FDB Formatster
2c788c233d apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-08-27 17:07:47 -07:00
Steve Atherton
e5e8a56b66 Merge branch 'master' of https://github.com/apple/foundationdb into debugging-cleanup
# Conflicts:
#	fdbbackup/FileDecoder.actor.cpp
2021-08-09 23:36:57 -07:00
Steve Atherton
2bf7b9b7a9 Remove many calls to .toString(), .printable(), and printable(...) from TraceEvent .detail() values to avoid double-escaping characters such that the same values look different in different events and can't be easily searched for. Also fixed a few warnings found by clang. 2021-08-08 00:04:54 -07:00
Jingyu Zhou
5346213e7f Fix trace event UnfishedBlocks 2021-08-02 09:26:42 -07:00
Jingyu Zhou
31a64106fb Avoid memory copies after decoding a new file block 2021-08-02 09:21:33 -07:00
Jingyu Zhou
426e906a87 Remove duplicated code 2021-08-01 22:39:28 -07:00
Jingyu Zhou
8f1946f522 Remove unused code 2021-08-01 21:55:39 -07:00
Jingyu Zhou
1a97c3942a Fix trace events 2021-08-01 19:25:39 -07:00
Jingyu Zhou
4f5bcaf050 Remove verbose stdout messages 2021-08-01 19:11:23 -07:00
Jingyu Zhou
e3629ef356 Refactor decoder 2021-08-01 19:09:09 -07:00
Jingyu Zhou
d7ac5830e6 Fix decoder bug of shadowing loop index 2021-08-01 13:48:45 -07:00
Jingyu Zhou
de1e9000a2 Add --hex_prefix flag 2021-07-31 21:08:13 -07:00
Jingyu Zhou
2282d8455a Fix trace format json 2021-07-31 19:02:33 -07:00
Meng Xu
9e4bee9e30 Handle ascii special characters 2021-08-01 01:50:33 +00:00
Jingyu Zhou
b930faf520 Log prefix filter 2021-07-31 12:52:39 -07:00
Jingyu Zhou
1145087543 Add UID to the events for decoders 2021-07-31 12:45:47 -07:00
Jingyu Zhou
0a98c76747 Fix trace event throttling by using different types 2021-07-31 12:24:59 -07:00
Jingyu Zhou
7054bf7054 Set max field length for long mutations 2021-07-31 11:28:13 -07:00
Jingyu Zhou
0ea432a638 Add traceevent for all printed mutations 2021-07-30 18:34:13 -07:00
Jingyu Zhou
52368eafbc Filter by version for mutations as well 2021-07-30 17:45:21 -07:00
Jingyu Zhou
1a4f6a4ccf Implement prefix key filter 2021-07-30 16:31:53 -07:00
Jingyu Zhou
c21fadeaea Add begin and end version filtering for files 2021-07-30 15:58:22 -07:00
sfc-gh-tclinkenbeard
c74047c665 Merge remote-tracking branch 'origin/master' into fix-more-clang-warnings 2021-07-28 11:51:02 -07:00
Steve Atherton
507c1f11e3 Add .log() to bare TraceEvent() invocations without any .detail()s to avoid clang-tidy warning about immediate destruction of object without use. 2021-07-26 19:55:10 -07:00
sfc-gh-tclinkenbeard
b9a22a61ef Fix many -Wreorder-ctor warnings 2021-07-23 17:33:18 -07:00
Steve Atherton
f596a81073 Rename ::TRUE and ::FALSE in BooleanParams to ::True and ::False so as to not conflict with the TRUE and FALSE macros provided by the Windows and MacOS SDKs. 2021-07-17 00:11:40 -07:00
sfc-gh-tclinkenbeard
79ff07a071 Added *BOOLEAN_PARAM macros to enforce documentation of boolean parameters 2021-07-02 15:04:42 -07:00
Jingyu Zhou
0ce3ed93ec Refactor TLS config for backup 2021-04-08 21:56:02 -07:00
Jingyu Zhou
86482606bb Add blob credentials option for fdbdecode 2021-04-08 21:56:02 -07:00
Jingyu Zhou
a5841dad7b Add TLS support to fdbdecode 2021-04-08 21:56:02 -07:00
Jingyu Zhou
219c8d8526 Remove verbose help output 2021-04-08 21:56:02 -07:00
Jingyu Zhou
7652bd1c08 Update help with all command line options 2021-04-08 21:56:02 -07:00
FDB Formatster
df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Markus Pilman
074531309b
Merge pull request #3769 from sfc-gh-ljoswiak/binary-info
Add CLI argument to print build information
2021-01-26 10:29:57 -07:00
Lukas Joswiak
f011af14bc Add build flags option to fdbdecode binary 2021-01-19 11:21:21 -08:00
sfc-gh-tclinkenbeard
86c7c1e946 Fix IAsyncFileSystem method signatures 2020-12-28 01:57:42 -04:00
sfc-gh-tclinkenbeard
b7f4d69dc4 Added VersionedKVPart class to FileDecoder.actor.cpp 2020-12-26 21:53:14 -04:00
A.J. Beamon
d128252e90 Merge release-6.3 into master 2020-05-22 09:25:32 -07:00
Jingyu Zhou
3c76722504 Fix valgrind error of invalid memory access 2020-03-30 16:38:22 -07:00
Jingyu Zhou
3c32835cce Fix decoder for unfinished version batch in a log
A mutation log's version batch data can be split into multiple blocks, and some
of the blocks can be spread across two mutation logs. Thus, the decoder needs
copy unfinished version batch data from previous file progress to the next one.
2020-03-30 11:34:51 -07:00
Jingyu Zhou
7831bec2b0 Address review comments on trace events 2020-03-24 10:54:12 -07:00
Jingyu Zhou
b18f192831 Fix decode bug of missing mutations
After reading a new block, all mutations are sorted by version again, which
can invalidate previously tuple. As a result, the decoded file will miss some
of the mutations.
2020-03-20 20:15:09 -07:00
Jingyu Zhou
d0a24dd20d Decode out of order mutations in old mutation logs
In the old mutation logs, a version's mutations are serialized as a buffer.
Then the buffer is split into smaller chunks, e.g., 10000 bytes each. When
writting chunks to the final mutation log file, these chunks can be flushed
out of order. For instance, the (version, chunck_part) can be in the order of
(3, 0), (4, 0), (3, 1). As a result, the decoder must read forward to find all
chunks of data for a version.

Another complication is that the files are organized into blocks, where (3, 1)
can be in a subsequent block. This change checks the value size for each
version, if the size is smaller than the right size, the decoder will look
for the missing chucks in the next block.
2020-03-20 20:15:08 -07:00
Jingyu Zhou
e15015ee6c Add mutation log version names
I.e., BACKUP_AGENT_MLOG_VERSION for 2001 and PARTITIONED_MLOG_VERSION for 4110.
2020-03-20 20:13:38 -07:00
Andrew Noyes
09f3690f09 Fix OPEN_FOR_IDE build 2020-02-03 10:42:05 -08:00
Jingyu Zhou
690e93145e Fix some comments 2020-01-22 19:38:46 -08:00