mirror of
https://github.com/apple/foundationdb.git
synced 2025-06-02 19:25:52 +08:00
Only truncate DiskQueues down to TLOG_HARD_LIMIT*2.
DiskQueue shrinking was implemented for spill-by-reference, as now a DiskQueue could grow "unboundedly" large. Without a minimum file size, write burst workloads would cause the DiskQueue to shrink down to 100MB, and then grow back to its usual ~4GB size in a cycle. File growth means filesystem metadata mutations, which we'd prefer to avoid if possible since they're more unpredicatble in terms of latency. In a healthy cluster, the TLog never spills, so the disk of a single DiskQueue file should stay less than 2*TLOG_SPILL_THRESHOLD. In the worst case of spill-by-value, the DiskQueue could grow to 2*TLOG_HARD_LIMIT. Therefore, having this limit will cause DiskQueue shrinking to never behave sub-optimally for spill-by-value, and will cause the DiskQueue files to return to the optimal size with spill-by-reference.
This commit is contained in:
parent
a269a784cc
commit
36dfbf4fb3
@ -304,9 +304,10 @@ public:
|
||||
self->writingPos = 0;
|
||||
|
||||
const int64_t activeDataVolume = pageCeiling(self->files[0].size - self->files[0].popped + self->fileExtensionBytes + self->fileShrinkBytes);
|
||||
if (self->files[1].size > activeDataVolume) {
|
||||
const int64_t desiredMaxFileSize = std::max( activeDataVolume, SERVER_KNOBS->TLOG_HARD_LIMIT_BYTES * 2 );
|
||||
if (self->files[1].size > desiredMaxFileSize) {
|
||||
// Either shrink self->files[1] to the size of self->files[0], or chop off fileShrinkBytes
|
||||
int64_t maxShrink = std::max( pageFloor(self->files[1].size - activeDataVolume), self->fileShrinkBytes );
|
||||
int64_t maxShrink = std::max( pageFloor(self->files[1].size - desiredMaxFileSize), self->fileShrinkBytes );
|
||||
self->files[1].size -= maxShrink;
|
||||
waitfor.push_back( self->files[1].f->truncate( self->files[1].size ) );
|
||||
}
|
||||
|
Loading…
x
Reference in New Issue
Block a user