book/rocksdb
1
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2025-05-14 00:43:07 +08:00
Code Issues Packages Projects Releases Wiki Activity

LOG more info on oldest snapshot and sequence numbers (#10454)

Browse Source 
Summary:
The info LOG file does not currently give any direct
information about the existence of old, live snapshots, nor how to
estimate wall time from a sequence number within the scope of LOG
history. This change addresses both with:
* Logging smallest and largest seqnos for generated SST files, which can
help associate sequence numbers with write time (based on flushes).
* Logging oldest_snapshot_seqno for each compaction, which (along with
that seqno info) helps us to determine how much old data might be kept
around for old (leaked?) snapshots. Including the date here I thought might
be excessive.

I wanted to log the date and seqno of the oldest snapshot with periodic
stats, but the current structure of the code doesn't really support that
because `DumpDBStats` doesn't have access to the DB object.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/10454

Test Plan:
manual inspect LOG from
`KEEP_DB=1 ./db_basic_test --gtest_filter=*CompactBetweenSnapshots*`

Reviewed By: ajkr

Differential Revision: D38326948

Pulled By: pdillinger

fbshipit-source-id: 294918ffc04a419844146cd826045321b4d5c038
This commit is contained in:
Peter Dillinger 2022-08-12 13:08:50 -07:00 committed by Facebook GitHub Bot
parent 2297769b38
commit 9fa5c146d7
2 changed files with 6 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
db/compaction/compaction_job.cc
View File

@ -1961,7 +1961,10 @@ void CompactionJob::LogCompaction() {
stream.EndArray();
}
stream << "score" << compaction->score() << "input_data_size"
<< compaction->CalculateTotalInputSize();
<< compaction->CalculateTotalInputSize() << "oldest_snapshot_seqno"
<< (existing_snapshots_.empty()
? int64_t{-1} // Use -1 for "none"
: static_cast<int64_t>(existing_snapshots_[0]));
}
}

3
db/event_helpers.cc
View File

@ -97,7 +97,8 @@ void EventHelpers::LogAndNotifyTableFileCreationFinished(
<< "file_number" << fd.GetNumber() << "file_size"
<< fd.GetFileSize() << "file_checksum"
<< Slice(file_checksum).ToString(true) << "file_checksum_func_name"
<< file_checksum_func_name;
<< file_checksum_func_name << "smallest_seqno" << fd.smallest_seqno
<< "largest_seqno" << fd.largest_seqno;
// table_properties
{