apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-06-02 11:15:50 +08:00
Code Issues Packages Projects Releases Wiki Activity
25,809 Commits 35 Branches 318 Tags
Commit Graph

79 Commits

Author SHA1 Message Date
Jay Zhuang
0efd403e59 Add inplace encryption/decryption API 2023-03-23 15:26:22 -07:00
Aaron Molitor
7cadc5420e update target_link with new path from build image 2023-03-03 16:30:07 -06:00
Russell Sears
ac9bc2738b Fix flowbench build in CI (requires new build image) 2023-02-27 15:38:58 -06:00
Russell Sears
f58d58ca77 Add support for pre-built googlebenchmark 2023-02-27 15:38:58 -06:00
Dan Adkins
be70a341f2 Revert "Use real clock source for trace events in real fdbserver, but now() in simulation. (#9270)" 
This reverts commit 02f78500b50a4e29fae00965fc7509b6966c8353.

That changed the default behavior of the clock used for trace logs, which
might break some tooling or workflows.
 2023-02-02 10:30:31 -08:00
Dan Adkins
02f78500b5
Use real clock source for trace events in real fdbserver, but now() in simulation. (#9270) 
CommitDebug trace events are useful for measuring, in detail, the time spent in
the various parts of a single transaction. Like all log events, they have a time
associated with them. This time comes for now(), which in a real fdb system is
only updated in the run loop. This renders the timestamps inaccurate in certain
CPU bound sections which don't have a wait, e.g. in the resolver.

We want to preserve the current behavior in simulation, where the timestamps are
artificial, deterministic between runs, and only updated in the run loop.

In a real system, we prefer to use a real clock so we can use the difference
between two trace events in the logs as a measurement of elapsed time.

This does not modify the behavior of other parts of the system, which use the
cached now() for various purposes.
 2023-02-02 12:33:45 -05:00
sfc-gh-tclinkenbeard
3c97f43138 Change Histogram::Unit::microseconds to milliseconds 2022-11-21 08:03:56 -08:00
Sam Gwydir
99d4bacf5d Merge remote-tracking branch 'origin/main' into ddsketch 2022-11-15 13:19:42 -08:00
Hui Liu
73d3e0f42f
Merge pull request #8775 from sfc-gh-huliu/bench 
Add micro-benchmark for blob delta serialization
 2022-11-14 11:37:39 -08:00
Sam Gwydir
23706c957b Use DDSketch for Sample Data. 2022-11-12 13:45:46 -08:00
Steve Atherton
e5e4457c6e Merge commit '8ad98dc9db2a1f9c3c1b44b22e0532bfa8c89ee5' into pml-delay 
# Conflicts:
#	fdbserver/storageserver.actor.cpp
 2022-11-11 11:49:31 -08:00
Hui Liu
0f8e5af5cf Add micro-benchmark for blob delta serialization 2022-11-11 09:22:11 -08:00
Steve Atherton
3b4a467a92 Remove runners list from PriorityMultiLock and rely on reference counting in the release handler instead of canceling the release handlers. This improves the microbenchmark by 26%. 2022-11-11 00:34:03 -08:00
Steve Atherton
9e9cf3ed8f Optimize hot paths and high churn structures in PML. Add second benchmark argument for inactive priorities to measure their cost impact. 2022-11-08 01:29:33 -08:00
Steve Atherton
70e078718c Bug fix in items processed count, changed delay/yield test to be a template. 2022-10-31 12:49:38 -07:00
Steve Atherton
34c61580a0 Added benchmark for delay(0) and yield(). 2022-10-30 20:50:58 -07:00
Steve Atherton
e5a5ec36a4 Merge commit '0872cbfb2f00886817f18584d95af217e28ad51d' into storageserver-pml 
# Conflicts:
#	fdbserver/storageserver.actor.cpp
 2022-10-19 13:25:31 -07:00
Andrew Noyes
8c22b7b779 Performance improvements and benchmark tweaks 2022-10-12 13:53:11 -07:00
Andrew Noyes
cf48ae206f Benchmark adding idempotency id mutations for commit batch 2022-10-12 11:29:09 -07:00
Hui Liu
a4c73a5f0a add benchmark for zstd 2022-10-11 08:23:55 -07:00
Steve Atherton
3228afefd3 Unrevert #7578 - storage server PriorityMultiLock and PML rewrite. 2022-10-06 23:41:28 -07:00
Markus Pilman
5774249e5b
Revert "[DRAFT] Redwood PriorityMultiLock enable different launch limits to be specified based on different priority level." 2022-09-23 12:22:47 -06:00
Steve Atherton
04b4960786 Merge branch 'main' into fzhao/RedwoodIOLaunchLimit 
# Conflicts:
#	fdbserver/VersionedBTree.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/ReadWrite.actor.cpp
 2022-09-22 00:39:51 -07:00
Steve Atherton
891cf87811 Added comments to PriorityMultiLock benchmark. 2022-09-21 00:55:35 -07:00
Steve Atherton
b607ca68ef Added BenchPriorityMultiLock benchmark. 2022-09-21 00:09:02 -07:00
Steve Atherton
ab41da174c Completely rewrote PriorityMultiLock scheduling and added a unit test for it. 2022-09-20 00:45:29 -07:00
A.J. Beamon
4fd64630e8 Convert literal string ref instances to use _sr suffix 2022-09-19 11:35:58 -07:00
Steve Atherton
a2e74af900
Merge pull request #8164 from sfc-gh-satherton/read-latency-logging-improvements 
Add sampling class tests to FlowBench and remove redundant tests from Redwood
 2022-09-12 18:05:22 -07:00
Steve Atherton
3063061c09 Move sample benchmarks to Flowbench. 2022-09-12 01:31:09 -07:00
sfc-gh-tclinkenbeard
7a870d395a Use faster KeepRunning loops in flowbench 2022-09-11 20:32:28 -07:00
Junhyun Shim
0874e553cd Place generateRandomData() under {I|Deterministic}Random 2022-07-20 13:21:11 +02:00
Sreenath Bodagala
c366adb4b7 - Benchmark for version vector serialization/deserialization code 2022-07-05 16:37:49 +00:00
Markus Pilman
a2ee14d3fb fixed flowbench 2022-06-28 10:23:38 -06:00
Markus Pilman
03d913a1de Flow compiling 2022-06-27 17:05:55 -06:00
Markus Pilman
ffaf15c12a moved wellknownendpoints and fixed some includes 2022-06-23 17:03:53 -06:00
Lukas Joswiak
88557d9169 Simplify function call when transaction is null 2022-06-22 14:50:17 -07:00
sfc-gh-tclinkenbeard
8ea68154bf Remove WITH_TLS CMake variable 2022-05-02 22:45:00 -07:00
Jingyu Zhou
cfcf0f152c Merge branch 'main-4a085fc84' into vv 
Fix Conflicts:
	fdbclient/NativeAPI.actor.cpp
	fdbserver/ClusterRecovery.actor.cpp
	fdbserver/MasterInterface.h
	fdbserver/masterserver.actor.cpp
	flow/error_definitions.h
 2022-03-30 22:28:06 -07:00
sfc-gh-tclinkenbeard
a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
Jingyu Zhou
fe0c137975 Add version vector microbenchmark 
This one measures cost of "getDelta" calls:

flowbench --benchmark_filter=bench_vv
2022-03-08T22:01:42+00:00
Running ./cbuild_output/bin/flowbench
Run on (32 X 1789.72 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x16)
  L1 Instruction 32 KiB (x16)
  L2 Unified 1024 KiB (x16)
  L3 Unified 36608 KiB (x1)
Load Average: 0.33, 0.51, 1.05
--------------------------------------------------------------------------------------
Benchmark                            Time             CPU   Iterations UserCounters...
--------------------------------------------------------------------------------------
bench_vv_getdelta/16/1            33.9 ns         33.9 ns     20604111 Tags=16 getDeltaTimes=1 items_per_second=29.514M/s
bench_vv_getdelta/64/1            33.5 ns         33.5 ns     20989482 Tags=64 getDeltaTimes=1 items_per_second=29.8637M/s
bench_vv_getdelta/512/1           36.5 ns         36.5 ns     19703013 Tags=512 getDeltaTimes=1 items_per_second=27.3722M/s
bench_vv_getdelta/1024/1          35.2 ns         35.2 ns     19945561 Tags=1024 getDeltaTimes=1 items_per_second=28.4083M/s
bench_vv_getdelta/16/8            6340 ns         6340 ns       110117 Tags=16 getDeltaTimes=8 items_per_second=1.26184M/s
bench_vv_getdelta/64/8            7257 ns         7257 ns        96336 Tags=64 getDeltaTimes=8 items_per_second=1.10235M/s
bench_vv_getdelta/512/8          13844 ns        13844 ns        51522 Tags=512 getDeltaTimes=8 items_per_second=577.871k/s
bench_vv_getdelta/1024/8         34612 ns        34612 ns        20470 Tags=1024 getDeltaTimes=8 items_per_second=231.137k/s
bench_vv_getdelta/16/64         200514 ns       200513 ns         3498 Tags=16 getDeltaTimes=64 items_per_second=319.182k/s
bench_vv_getdelta/64/64         500066 ns       500057 ns         1000 Tags=64 getDeltaTimes=64 items_per_second=127.985k/s
bench_vv_getdelta/512/64        570166 ns       570131 ns         1222 Tags=512 getDeltaTimes=64 items_per_second=112.255k/s
bench_vv_getdelta/1024/64       747168 ns       747156 ns          939 Tags=1024 getDeltaTimes=64 items_per_second=85.6581k/s
bench_vv_getdelta/16/512       1749315 ns      1749305 ns          401 Tags=16 getDeltaTimes=512 items_per_second=292.688k/s
bench_vv_getdelta/64/512       6656289 ns      6656249 ns          105 Tags=64 getDeltaTimes=512 items_per_second=76.9202k/s
bench_vv_getdelta/512/512     36588584 ns     36588013 ns           19 Tags=512 getDeltaTimes=512 items_per_second=13.9937k/s
bench_vv_getdelta/1024/512    37691398 ns     37691154 ns           19 Tags=1024 getDeltaTimes=512 items_per_second=13.5841k/s
bench_vv_getdelta/16/1024      3616253 ns      3616233 ns          193 Tags=16 getDeltaTimes=1024 items_per_second=283.168k/s
bench_vv_getdelta/64/1024     13715322 ns     13715233 ns           51 Tags=64 getDeltaTimes=1024 items_per_second=74.6615k/s
bench_vv_getdelta/512/1024   112346008 ns    112344280 ns            6 Tags=512 getDeltaTimes=1024 items_per_second=9.11484k/s
bench_vv_getdelta/1024/1024  157170467 ns    157169544 ns            4 Tags=1024 getDeltaTimes=1024 items_per_second=6.51526k/s
 2022-03-08 15:36:32 -08:00
sfc-gh-tclinkenbeard
a32547cbbe Merge remote-tracking branch 'origin/main' into flow-enhancements 2022-02-15 15:56:42 -08:00
Ata E Husain Bohra
591ef57857
Upgrade AES 128 GCM -> AES 256, StreamCipher code refactor (#6314) 
* Upgrade AES 128 GCM -> AES 256, StreamCipher code refactor

Major changes proposed are:
1. Refactor StreamCipher code to enable instantiation of
   multiple encryption keys. However, code still retains
   a globalEncryption key semantics used in Backup file
   encryption usecase.
2. Enhance StreamCipher to provide HMAC signature digest
   generation. Further, the class implements HMAC encryption
   key derivation function.
3. Upgrade StreamCipher to use AES 256 GCM mode from currently
   supported AES 128 GCM mode.
   Note: The code changes the encryption key size, however, the
         feature is NOT currently in use, hence, should be OK.
3. Add EncryptionOps validation and benchmark toml supported
   workload, it does the following:
   a. Allow user to configure encrypt-decrypt of a fixed size
      buffer or variable size buffer [100, 512K]
   b. Allow user to configure number of interactions of the runs,
      in each iteration: generate random data, derive an encryption
      key using HMAC SHA256 method, encrypt data and
      then decrypt data. It collects following metrics:
    i) time taken to derive encryption key.
    ii) time taken to encrypt the buffer.
    iii) time taken to decrypt the buffer.
    iv) total bytes encrypted and/or decrypted
   c. Along with stats it basic basic validations on the encrypted
      and decrypted buffer
   d. On completion for test, records the above mentioned metrics
      in trace files.
 2022-01-31 19:52:44 -06:00
sfc-gh-tclinkenbeard
c834778d3d Apply clang-format to BenchNet2.actor.cpp 2022-01-23 23:49:58 -08:00
Jon Fu
476aaa73c6 modify benchmark to avoid threadsleep and change runloop to process IO properly 2021-12-16 16:20:22 -05:00
Jon Fu
169838cec3 stub changes for IO queue and attempt at new benchmark 2021-12-16 13:28:38 -05:00
sfc-gh-tclinkenbeard
a2bb14762a Some cleanup of BenchCallback.actor.cpp 2021-12-12 20:17:15 -08:00
sfc-gh-tclinkenbeard
c1b480bedd Add bench_net2 benchmark to flowbench 2021-12-12 20:17:15 -08:00
sfc-gh-tclinkenbeard
cde26d2d4f Add bench_callback to flowbench 2021-12-12 20:17:15 -08:00
Andrew Noyes
5b167729d2 Upgrade google benchmark to v1.6.0 
This fixes a compile error with clang 13
 2021-12-02 16:06:40 -08:00
Aaron Molitor
a930039952 remove (cmake version and git detached head) warnings from build logs 2021-11-29 15:11:20 -08:00