21238 Commits

Author SHA1 Message Date
sfc-gh-tclinkenbeard
20ac60fb11 Set throttling ratio in GlobalTagThrottler::tryUpdateAutoThrottling 2022-07-19 17:04:04 -07:00
sfc-gh-tclinkenbeard
f41515e66b Implement GlobalTagThrottler::busy*TagCount methods 2022-07-19 17:04:01 -07:00
sfc-gh-tclinkenbeard
afba51b7f8 Add tracing to GlobalTagThrottler 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard
fe05cc5c72 Update busy read tag reporting in status json 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard
145d10e3a4 Compute cluster-wide average transaction cost for each tag only once in getClientRates 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard
080518d014 Use getQuota method in GlobalTagThrottlerImpl::getDesiredTps 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard
d0851a67e2 Update TransactionTagCounter::startNewInterval tracing 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard
2306757e90 Fix GlobalTagThrottler::autoThrottleCount 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard
4977805d8e Fix failing global tag throttling unit tests 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard
c86ae6cee5 Fix /GlobalTagThrottler/Simple unit test 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard
e128c517d3 New GlobalTagThrottler implementation tracking more per-storage
server metrics
2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard
6a63db08c8 Added /GlobalTagThrottler/ReservedWriteQuota unit test 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard
5b0eb135e0 Guarantee reserved quota in GlobalTagThrottler 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard
8e08033974 Add /GlobalTagThrottler/MultiTagActiveThrottling unit test 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard
bac583ef0c Add new active throttling unit test for global tag throttler 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard
34a78a82f1 Add MockStorageServer class for global tag throttler testing 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard
88ca6376ce Add LimitType and OpType enum classes for global tag throttler 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard
fb6540ab43 Add throttling ratio to GlobalTagThrottler 2022-07-19 16:29:10 -07:00
sfc-gh-tclinkenbeard
ab2110142a Add /GlobalTagThrottler/RemoveQuota unit test 2022-07-19 16:29:10 -07:00
sfc-gh-tclinkenbeard
019900be67 Add GlobalTagThrottlerImpl::removeUnseenTags 2022-07-19 16:29:10 -07:00
sfc-gh-tclinkenbeard
3161bcfa52 Remove outdated TODO and FIXME comments from GlobalTagThrottlerImpl 2022-07-19 16:29:06 -07:00
sfc-gh-tclinkenbeard
b49c36f0b0 Add StorageQueueInfo::getWriteQueueSizeLimitRatio method 2022-07-19 16:28:27 -07:00
Jingyu Zhou
bc62b538b1
Merge pull request #7627 from jzhou77/remove-metrics
Remove unused "readQueueSizeMetric"
2022-07-19 15:55:52 -07:00
Dan Lambright
7e296c2367
simple replication (#7625) 2022-07-19 18:30:34 -04:00
Jingyu Zhou
fde5aeeb62 Remove unused "readQueueSizeMetric" 2022-07-19 14:38:10 -07:00
Xiaoxi Wang
dea775ad0d
Merge pull request #7580 from sfc-gh-xwang/doc/readaware
update data distributor design doc
2022-07-19 13:29:12 -07:00
Markus Pilman
1de37afd52
Make TEST macros C++ only (#7558)
* proof of concept

* use code-probe instead of test

* code probe working on gcc

* code probe implemented

* renamed TestProbe to CodeProbe

* fixed refactoring typo

* support filtered output

* print probes at end of simulation

* fix missed probes print

* fix deduplication

* Fix refactoring issues

* revert bad refactor

* make sure file paths are relative

* fix more wrong refactor changes
2022-07-19 13:15:51 -07:00
Xiaoxi Wang
9dfd58978f
Apply suggestions from code review
Co-authored-by: Jingyu Zhou <jingyuzhou@gmail.com>
2022-07-19 12:03:49 -07:00
A.J. Beamon
0d20cd2068
Merge pull request #7618 from sfc-gh-ajbeamon/fix-compile-warnings
Fix a few compile warnings about unused variables
2022-07-19 11:10:42 -07:00
A.J. Beamon
c4cd72bea1 Fix a few compile warnings about unused variables 2022-07-19 10:01:41 -07:00
A.J. Beamon
0e03fbd9e7
Merge pull request #7605 from sfc-gh-ajbeamon/fix-trace-log-detail-formatting
Remove some instances of using printable or toString in a trace event detail
2022-07-19 08:24:58 -07:00
Dan Lambright
775ab3323e
update tag client cache on getKeyLocation (#7613) 2022-07-19 11:24:17 -04:00
Trevor Clinkenbeard
1eec6f993d
Merge pull request #7572 from sfc-gh-akejriwal/tenantquota
Introduce storage quotas per tenant
2022-07-18 20:00:22 -07:00
A.J. Beamon
38855fd786 Remove some instances of using printable or toString in a trace event detail. 2022-07-18 16:24:33 -07:00
Ankita Kejriwal
2757262ec9
Simplify storageQuotaKey() function implementation
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
2022-07-18 15:56:11 -07:00
Renxuan Wang
570db7a7d2
Fix processDiskReadSeconds and processDiskWriteSeconds. (#7608) 2022-07-18 13:47:45 -07:00
Jingyu Zhou
d28fb2b809
Merge pull request #7607 from sfc-gh-jshim/dlopen-nodelete-for-sanitizer
Apply RTLD_NODELETE for USE_SANITIZER builds
2022-07-18 13:25:33 -07:00
Trevor Clinkenbeard
18a2c3f8a8
Merge pull request #7489 from sfc-gh-tclinkenbeard/cstate-pimpl
Mark `CoordinatedState::getConflict` const
2022-07-18 13:22:18 -07:00
Junhyun Shim
8b70c7050d Apply RTLD_NODELETE for USE_SANITIZER builds
This allows their stack trace to be symbolized in CI/Jenkins XSAN runs
2022-07-18 19:25:12 +02:00
Andrew Noyes
989e323991
Attempt to fix windows build (#7600)
* Attempt to fix windows build

The windows build is complaining that some symbols in flow are
duplicate, which seems fair since we're currently compiling all the flow
sources and _also_ linking to flow for flowlinktest. This change makes
it so that we only link flow, and don't compile the flow source files
again.

* Remove source files from fdbclient and fdbrpc link tests
2022-07-18 08:59:03 -07:00
Ankita Kejriwal
882df774c0 Storage quota: use transaction options and minor fixes in the test 2022-07-15 16:34:04 -07:00
A.J. Beamon
02ab3b05ab
Merge pull request #7590 from sfc-gh-ajbeamon/update-prefix-encoding-in-tenant-metadata-json
Use a more standard encoding in the JSON metadata reported for tenants
2022-07-15 14:32:53 -07:00
Josh Slocum
fb4fc8af9d
Merge pull request #7587 from sfc-gh-ahusain/ahusain-hybrid_encryption_active
BlobGranule encryption and compression support
2022-07-15 15:57:51 -05:00
Ata E Husain Bohra
05e7f34c80 BlobFile Encryption and compression support
Fix windows CI build issue.

Description

Testing
2022-07-15 11:04:02 -07:00
He Liu
4dc742493e
Add cleanUpChangeFeeds(). (#7598)
Co-authored-by: He Liu <heliu@apple.com>
2022-07-15 10:29:09 -07:00
Ata E Husain Bohra
1da88d5d8c BlobFile Encryption and compression support
Fix formatting issues

Description

Testing
2022-07-14 22:46:14 -07:00
Ata E Husain Bohra
f288abebc2 BlobFile Encryption and compression support
Fix formatting issues and rename KNOB

Description

Testing
2022-07-14 17:22:00 -07:00
Ankita Kejriwal
6f130b4a69 Change quota type to unit64_t to avoid macOS compilation issue 2022-07-14 17:09:03 -07:00
Ata E Husain Bohra
f6f117592d BlobFile Encryption and compression support
- Limit verbose logging under DEBUG_MACRO
 - Update/Add code documentation

Description

Testing
2022-07-14 17:04:14 -07:00
Ata E Husain Bohra
3a3811d6cc BlobFile Encryption and compression support
Few fixes
 - Update BlobGranuleCipherKeysMeta to use 'std::string' to
   persist 'Initialization Vector'
 - Update WriteSnapshot to persist 'BlobGranuleCipherKeysMeta' as part
   of BlobFileIndex

Description

Testing
2022-07-14 17:04:14 -07:00