9 Commits

Author SHA1 Message Date
Markus Pilman
a47ed89018 Linux fixes and addressed review comments 2022-06-23 20:52:13 -06:00
Kevin Hoxha
e579038018
Add DDSketch to mako (#7167)
* Add logic for DDSketch in mako

* Return double from percentile() and fix crash in deserialize()

* make sure to serialize and print result from mergeSketchReport()

* clean up comments

* move ddsketch into its own file

* remove LatencySampleBin and add DDSketch to ThreadStatistics

* Update DDSketch implementation

* remove assertions that cause circular references

* add DDSketchMako as a subsclass from DDSketch

* Merge branch 'ddsketch_mako' of github.com:sfc-gh-khoxha/foundationdb into ddsketch_mako

* Revert "Merge branch 'ddsketch_mako' of github.com:sfc-gh-khoxha/foundationdb into ddsketch_mako"

This reverts commit cc29a68aefd1b385b563bfbaa09a32e399c0d233.

* add ddsketch mako class and rename export flag

* remove redundant decimal roundings

* print max/min/avg from ddsketch

* remove latency sample bin completly

* Make ThreadStatistics dump latency to a file and read from file in printReport()

* make sure to add latency data from file to final stats

* change mergeSketchReport to use new ThreadStatistics serialization (1)

* use C-style string arrays in Arguments instead of std::string

* remove unused header

* only serialize non-empty sketches

* fix CentOS build error

* Update report file count properly

* avoid deserializing empty sketches

* fix segmentation fault when getting file name for export_sketch_path

* make sure to properly add file to report_files list

* fix printing bugs when running in report mode

* fix incorrect insertion of report files

* don't use range based loop for char array

* don't reset args.num_report_files

* Update the usage info for new options

* switch to using std::vector for sketches instead of std::array

* make sure to use true/false instead of 1/0 for booleans

* remove op_name if not being used

* remove fp code in dumpThreadSamples

* replace lambda with function in printReport

* merge and print stats in seperate functions

* make sure to exit after printing report

* address review feedback

* make defaultMin, defaultMax static and move setBucketSize to protected

* switch to reverse iterators when moving backwards along bucket array
2022-06-06 18:19:31 -07:00
Junhyun Shim
2f9506f5cd Introduce hard cap to latency sampling & apply clang format 2022-04-26 11:19:48 +02:00
Junhyun Shim
81211989f1 Disable new sample collection after first bad_alloc
- Add more info to --async_xacts description
- Clean up unused macro in fdb_api.hpp
2022-04-26 10:54:39 +02:00
Junhyun Shim
c650d5366f Prefix sources and headers with license comment
Add include guards
2022-04-13 19:21:36 +02:00
Junhyun Shim
00ce4f6248 Add asynchronous execution mode
- option -t (or --threads) determines the number of worker threads
  running asio job scheduler (io_context).
- value of --async_xacts indicates how many concurrent transactions should be running
  (only applies to '-m build' or '-m run')
- concurrent transactions whose count is specified in --async_xacts
  walks through the workload sequence (-x) as worker threads would in
  synchronous execution mode: i.e. --async_xacts=0.
- concurrent transactions in async mode, as would worker threads in sync mode,
  divide up the insertion range in build mode,
  and iteration limit in 'run' mode.
2022-03-22 22:00:37 +01:00
Junhyun Shim
cc2b9fad1e Simpler logging with thread-local loggers
- No need to repeat parts of the messages: e.g. "ERROR: "
- Thread-specific information prefixed before messages: e.g. "WORKER[  1: 4]"
- Misc. C-to-CPP styling
2022-03-19 22:53:19 +01:00
Junhyun Shim
8c0d3e9469 Use camelCase for FDB API & Mako classes 2022-03-07 06:19:33 +01:00
Junhyun Shim
bcf52e6760 Style changes: C > C++
* more controlled shared memory access
* simplified thread arguments
* use namespace 'mako'
* separate headers by their concerns (time, statistics, shared memory)
* introduce Stopwatch class
2022-03-07 03:52:41 +01:00