57 Commits

Author SHA1 Message Date
Jingyu Zhou
b18f192831 Fix decode bug of missing mutations
After reading a new block, all mutations are sorted by version again, which
can invalidate previously tuple. As a result, the decoded file will miss some
of the mutations.
2020-03-20 20:15:09 -07:00
Jingyu Zhou
d0a24dd20d Decode out of order mutations in old mutation logs
In the old mutation logs, a version's mutations are serialized as a buffer.
Then the buffer is split into smaller chunks, e.g., 10000 bytes each. When
writting chunks to the final mutation log file, these chunks can be flushed
out of order. For instance, the (version, chunck_part) can be in the order of
(3, 0), (4, 0), (3, 1). As a result, the decoder must read forward to find all
chunks of data for a version.

Another complication is that the files are organized into blocks, where (3, 1)
can be in a subsequent block. This change checks the value size for each
version, if the size is smaller than the right size, the decoder will look
for the missing chucks in the next block.
2020-03-20 20:15:08 -07:00
Jingyu Zhou
e15015ee6c Add mutation log version names
I.e., BACKUP_AGENT_MLOG_VERSION for 2001 and PARTITIONED_MLOG_VERSION for 4110.
2020-03-20 20:13:38 -07:00
Andrew Noyes
09f3690f09 Fix OPEN_FOR_IDE build 2020-02-03 10:42:05 -08:00
Jingyu Zhou
690e93145e Fix some comments 2020-01-22 19:38:46 -08:00
Jingyu Zhou
3854d9a49a Fix decoder to handle multi-part values
When a value (i.e., mutations for a version) is large, it will be split into
multiple key value pairs. This is not handled previously and fixing it also
consolidate the interface of DecodeProgress.
2020-01-22 19:38:46 -08:00
Jingyu Zhou
7d1b9fe6d3 Add mutation file decoder 2020-01-22 19:38:46 -08:00