apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-06-02 03:12:12 +08:00
Code Issues Packages Projects Releases Wiki Activity
20,209 Commits 35 Branches 318 Tags
c69a07a858
Commit Graph

66 Commits

Author SHA1 Message Date
Steve Atherton
702bd59ded Clear unused memory regions for structures written to disk. 2022-04-07 23:02:33 -07:00
Steve Atherton
3ef18bc173 Rename Page and VersionHeader to PageHeader for consistency. Check for known encrypted types explicitly in isEncodingTypeEncrypted(). Make writePhysicalBlock() hold a Page reference which is simpler and more robust. Remove unused argument in switchTree(). 2022-04-07 20:59:34 -07:00
Steve Atherton
23a27d78db Changed DeltaTree2::Cursor to hold a Reference<DecodeCache> instead of a pointer to enable more flexible DecodeCache lifetimes. 2022-04-04 03:38:50 -07:00
Steve Atherton
39fb0a44d7 Merge commit 'f09bdc840c00d712487500b9e752d87cedb1964a' into redwood-header-changes 
# Conflicts:
#	fdbserver/VersionedBTree.actor.cpp
 2022-04-03 00:37:01 -07:00
Steve Atherton
aa8ab494a2 Fix undefined behavior. 2022-03-26 19:37:09 -07:00
Steve Atherton
9f205217a4 Merge branch 'main' of https://github.com/apple/foundationdb into redwood-rare-bugs-6574 2022-03-21 21:57:57 -07:00
sfc-gh-tclinkenbeard
a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
sfc-gh-tclinkenbeard
0ae9ba5fd0 Add FastInaccurateEstimate boolean parameter for Arena::getSize 2022-03-16 16:54:27 -07:00
Steve Atherton
bade9a3ec3 Added toString() methods for DeltaTree::DecodeCache. 2022-03-10 00:07:22 -08:00
Steve Atherton
a32c3a2891 Redwood now uses one global page cache eviction order and size limit for all page sizes, for real or virtual processes, and DecodeCache memory is counted against the global page cache budget. 2022-02-25 03:20:53 -08:00
Steve Atherton
17fe322bc4 Added near real-time tracking of Redwood page DecodeCache memory. 2022-02-18 03:26:05 -08:00
Steve Atherton
a023d49947 Bug fix in DeltaTree - Insert() of record outside of lower/upper boundaries used for key decoding would calculate incorrect record delta. 2022-02-10 01:36:24 -08:00
Steve Atherton
7a08d59c22 Addressing PR discussions. Many renames. Changed encryption API to use new EncryptionKeyRef struct. Added a tree offset byte to BTreePage to allow for adding an additional field in the future. Added dummy definition for valgrind defined check. Added check that BTree pages are encrypted if encryption is enabled to prevent attack by replacing encrypted page with unencrypted page. 2022-01-06 22:27:22 -08:00
Steve Atherton
592006a537 Several bug fixes in ArenaPage pre and post read/write initialization. Several bug fixes in Queue and Page usage of ArenaPage. Bug fixes in page splitting logic involving incorrect space calculations within the new available page space. 2021-12-11 19:41:14 -08:00
Steve Atherton
4bc0cfe4af Many bug fixes in new page API and usages of it. 2021-12-09 02:21:50 -08:00
Steve Atherton
470896bdc4
Redwood inline same-size value updates () 
* Refactored mutation application in leaf nodes to do fewer comparisons and do in place value updates if the new value is the same size as the old value.

* Renamed updatingInPlace to updatingDeltaTree for clarity.  Inlined switchToLinearMerge() since it is only used in one place.
 2021-11-10 08:22:57 -08:00
Steve Atherton
5c85555ea9 Refactored mutation application in leaf nodes to do fewer comparisons and do in place value updates if the new value is the same size as the old value. 2021-11-06 21:07:00 -07:00
sfc-gh-tclinkenbeard
d0c9cf4fb0 Enable mismatched-tags clang warning 2021-11-01 14:18:31 -07:00
sfc-gh-tclinkenbeard
25257f6f87 Enable unused-function warning for clang 2021-11-01 14:18:31 -07:00
sfc-gh-tclinkenbeard
b9a22a61ef Fix many -Wreorder-ctor warnings 2021-07-23 17:33:18 -07:00
Steve Atherton
8cbc26d436 Added documentation for DeltaTree2. 2021-06-10 02:29:17 -07:00
Steve Atherton
69f7c7cba2 Make deltatree debug toggle easier to use. 2021-06-09 21:19:52 -07:00
Steve Atherton
f2904dadf3 Reading BTree pages no longer requires boundary records, as they are not needed if the page is already cached. 2021-06-08 16:58:35 -07:00
Steve Atherton
f95d592db8 Optimized record delta decoding / applying a bit, changed substring order to (value, keySuffix) since value is needed more frequently. Added DeltaTree2 item type requirement to create T from base's cached partial item instead of full record. 2021-06-08 16:58:35 -07:00
Steve Atherton
3451c2242f DeltaTree2::Cursor now reconstructs current item on-demand, caches it in an Optional member, and does not initialize it when a cursor is copied. 2021-06-08 16:58:35 -07:00
Steve Atherton
751bac2271 Write path no longer uses non-caching reads because it is no longer necessary to avoid a page copy. Page copies are only done just before an actual change is made. 2021-06-08 16:46:09 -07:00
Steve Atherton
a6f7d37a25 Bug fixes related to DeltaTree2::Cursor contract being different from DeltaTree::Cursor. 2021-06-08 16:46:09 -07:00
Steve Atherton
8ef516ead2 Bug fixes from bad search/replace. DeltaTree2::Cursor now keeps current decoded item as a member instead of calculating it on demand in get(). 2021-06-08 16:46:09 -07:00
Steve Atherton
d208d3f3ec Bug fixes, moveFirst/Last didn't handle tree size of 1 correctly. 2021-06-08 16:46:09 -07:00
Steve Atherton
caf4b3c345 DeltaTree2 refactor. Nodes no longer contain parent offsets. DecodedCache no longer uses a hash but rather a vector of DecodedNodes, which Cursors reference by vector index. DecodedNodes contain parent node indexes which are populated on-demand, making storage in the serialized form no longer necessary. 2021-06-08 16:46:04 -07:00
Steve Atherton
b0ec76d401 Test output improvements. 2021-06-08 15:32:54 -07:00
Steve Atherton
9ab69b5cb1 RedwoodRecordRef support for DeltaTree2. 2021-06-08 15:32:54 -07:00
Steve Atherton
701f05e513 Bug fix, recursive call to get() could cause rehashing so hash lookup must be redone afterward. 2021-06-08 15:32:54 -07:00
Steve Atherton
b2c7d957e2 Added DeltaTree2, which can be shared between updated versions of the same tree, but so far it is 50% slower. 2021-06-08 15:32:26 -07:00
Andrew Noyes
1ee25e9b91 Avoid casting NaN to uint8_t 2021-06-04 09:43:50 -07:00
Steve Atherton
390b026f08
Merge branch 'master' into arena-page 2021-05-16 05:06:50 -07:00
Steve Atherton
799e7cc213 BTreeCursor now exposes the current Page and Mirror arenas so that users extracting Standalone Keys and Values can just reference those instead of copying. 2021-05-05 21:05:32 -07:00
sfc-gh-ngoyal
63d82b9b63
Merge pull request from sfc-gh-satherton/redwood-improvements 
Redwood page splitting/building refactor
 2021-05-03 13:07:41 -07:00
Josh Slocum
4b5bca6761 Minor Redwood comparison optimizations 2021-04-23 18:49:43 +00:00
Steve Atherton
5c93e684f8 Added comments. 2021-04-04 19:23:08 -07:00
Steve Atherton
90ebf90c8b Refactored page rebuild logic to bulk build pages full and split pages more evenly. 2021-04-03 19:54:49 -07:00
FDB Formatster
df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Andrew Noyes
8766bb44b1 Address review comment 2021-03-09 01:40:59 +00:00
Andrew Noyes
5ecb1dfcba Move ASSERT to right before dereference 2021-03-08 22:37:30 +00:00
Andrew Noyes
7188c7ddfb Add ASSERT to help clang-tidy 
If n == nullptr here, then we'll dereference a nullptr. Add an assert.
 2021-03-08 18:50:28 +00:00
Steve Atherton
f306666e21 Added commonPrefixLength for StringRefs that takes a prefix skip length. 2020-05-10 04:24:11 -07:00
Steve Atherton
1966dbb576 Checkpointing large refactor of how commitSubtree() works regarding internal pages such that now they can be incrementally modified and detecting changes involves far fewer comparison and staging of records. 2020-05-08 07:11:51 -07:00
Steve Atherton
dda0993d16 Apply clang-format to Redwood source. 2020-04-24 14:12:40 -07:00
Steve Atherton
d4334256f1 Bug fix: BTree::writePages() was not accounting for the small amount of unpredictability in node overhead sizes which could lead to serializing a tree that doesn't fit into its destination page buffer. WritePages() now skips the prefix common to all records in the input set when calling deltaSize() to estimate resulting DeltaTree serialized sizes. 2020-04-08 20:18:41 -07:00
Steve Atherton
9bf0b5e71f RedwoodRecord simplified to remove chunked key-value pair concept. Cursor and InternalCursor updated to be correct (but not optimal) for new record definition. DeltaTree::seek* refactored to support more seek types and with minor code duplication. Lots of debug output improvements. Refactored how bulk pages are written and introduced fill factor percentage (though since internal pages are not yet incrementally modified this is temporarily worse for those). 2020-04-06 20:27:30 -07:00