1
0
mirror of https://github.com/apple/foundationdb.git synced 2025-06-01 18:56:00 +08:00

67 Commits

Author SHA1 Message Date
Josh Slocum
7f45cccb56
More granule purging fixes ()
* Granule purge cannot delete history entry for fully deleting granule until all children are completely done splitting

* Several purging fixes related to granule history

* Fixed typo in refactor

* fixing memory model for purgeRange

* formatting

* weakening granule purge test for now

* cleanup

* review comments
2022-08-03 16:43:27 -05:00
Josh Slocum
6115a406d4 formatting 2022-07-25 13:52:47 -05:00
Josh Slocum
af60e2ea32 Fixed granule purging bug and improved debugging for purging 2022-07-25 12:19:41 -05:00
Josh Slocum
ee1b0cdd43 making purge failures fail test 2022-07-23 13:10:03 -05:00
Josh Slocum
316b7a5344 Merge branch 'main' into granule_merging_converge 2022-07-20 12:13:48 -05:00
Josh Slocum
44885aa719 cleanup 2022-07-20 07:46:57 -05:00
Josh Slocum
78b6a96006 Merge branch 'main' into granule_merging_batch 2022-07-20 07:42:26 -05:00
Josh Slocum
12b6f386cb Refactoring granule flush to retry properly on granule rollback 2022-07-19 19:49:20 -05:00
Josh Slocum
340a01763b disabling merge convergence check for now 2022-07-18 16:09:14 -05:00
Josh Slocum
866dda5763 Adding check for merge convergence 2022-07-14 20:49:43 -05:00
Josh Slocum
0b0ac16a4c Merge branch 'main' into granule_merging 2022-07-12 09:09:30 -05:00
Markus Pilman
d35445a868 enforce include modularization in cmake 2022-06-23 14:37:35 -06:00
Josh Slocum
b3597ef3a8 Added plumbing for tenant-aware purge granules 2022-06-16 13:04:34 -05:00
Josh Slocum
d6920cde28 Implemented blob granule merging 2022-06-09 10:50:53 -05:00
Josh Slocum
567b1d35f7 Full integration with BlobConnectionProvider for blob granules 2022-06-01 12:39:51 -05:00
Josh Slocum
6276cebad9
Blob integration ()
* Fixing leaked stream with explicit notify failed before destructor

* better logic to prevent races in change feed fetching

* Found new race that makes assert incorrect

* handle server overloaded in initial read from fdb

* Handling more blob error types in granule retry

* Fixing rollback metadata problem, added better debugging

* Fixing version race when fetching change feed metadata

* Better racing split request handling

* fixing assert

* Handle change feed popped check in the blob worker

* fix: do not use a RYW transaction for a versionstamp because of randomize API version ()

* more merge conflict issues

* Change feed destroy fixes

* Fixing change feed destroy and move race

* Check error condition in BG file req

* Using relative endpoints for blob worker interface

* Fixing bug in previous fix

* More destroy and move race fixes

* Don't update empty version on destroy in case it gets rolled back. moved() and removing will take care of ensuring it is not read

* Bug fix ()

* fix: do not use a RYW transaction for a versionstamp because of randomize API version

* fix: if the initialSnapshotVersion was pruned, granule history was incorrect

* added a way to compress null bytes in printable()

* Fixing durability issue with moving and destroying change feeds

* Adding fix for not fully deleting files for a granule that child granules need to re-snapshot

* More destroy and move races

* Fixing change feed destroy and pop races

* Renaming bg prune to purge, and adding a C api and unit test for it

* more cleanup

* review comments

* Observability for granule purging

* better handling for change feed not registered

* Fixed purging bugs ()

* fix: do not use a RYW transaction for a versionstamp because of randomize API version

* fix: if the initialSnapshotVersion was pruned, granule history was incorrect

* added a way to compress null bytes in printable()

* fixed a few purging bugs

Co-authored-by: Evan Tschannen <evan.tschannen@snowflake.com>
2022-04-08 14:15:25 -07:00
Josh Slocum
cb918b9cef Added basic blob granule consistency check 2022-04-04 11:38:42 -05:00
Renxuan Wang
0a332ee1c1 Add proxy option to backup and restore params. 2022-03-28 17:10:49 -07:00
Josh Slocum
989dd8d7eb Finished implementing beginVersion 2022-03-24 17:15:11 -05:00
Josh Slocum
f27475e2f4 Merge branch 'main' into blob_integration 2022-03-22 11:41:58 -05:00
sfc-gh-tclinkenbeard
a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
Evan Tschannen
2c88a189a9 more pruning bug fixes 2022-03-15 13:34:59 -07:00
Evan Tschannen
d46e551f11 merge 2022-03-15 13:09:10 -07:00
Josh Slocum
c35e020da7 SS and BM test macros 2022-03-10 12:38:23 -06:00
Josh Slocum
d7367dad99 Simplifying CF gap validation to ignore SS CF fetches 2022-03-10 11:56:22 -06:00
Josh Slocum
b21d0943b9 client-focused cleanup 2022-03-09 10:01:25 -06:00
Josh Slocum
f230ac0e41 Fixing more merge issues 2022-03-09 09:13:03 -06:00
Josh Slocum
e71b3533f9 Merge branch 'main' into blob_integration 2022-03-09 08:59:56 -06:00
Jingyu Zhou
1a5bf25b5c Update code base to use fmt 8.1.1 2022-03-04 15:52:06 -08:00
Evan Tschannen
976375466f fixed more pruning bugs 2022-03-02 17:03:37 -08:00
Josh Slocum
a14f3ff067 Do some verify tests with no pruning 2022-03-01 12:52:26 -06:00
Evan Tschannen
6f1fb71949 Fixed a number of different pruning problems 2022-03-01 10:13:20 -08:00
Evan Tschannen
1990216e89 use global knowledge inside simulation to determine when it should be possible to get a too_old error 2022-02-22 13:03:57 -08:00
Evan Tschannen
330b2b48ec improved file cleanup execution and testing 2022-02-22 12:00:09 -08:00
Josh Slocum
f2b9eb1d4b Fixing potential bug of mixed-version FDB result in granule verifier 2022-01-31 10:44:09 -06:00
Evan Tschannen
966d9d9c15
Merge pull request from sfc-gh-etschannen/blob_integration
fix: do not report errors when the blob verifier is actor_cancelled
2022-01-21 12:06:34 -08:00
Evan Tschannen
fee9e932e7 fix: do not report errors when the blob verifier is actor_cancelled 2022-01-21 12:06:00 -08:00
Josh Slocum
7f9bce40a7 retrying GRVs in BlobGranuleVerifier 2022-01-21 13:36:56 -06:00
Josh Slocum
4d2650f4dc Fixed a couple issues with failures and the final availability check 2022-01-07 11:21:05 -06:00
Josh Slocum
807ec37e84 BG request retry on failed blob worker properly 2022-01-05 17:38:49 -06:00
Josh Slocum
bc69521a91 Several fixes with restarting BW/BM 2022-01-05 12:48:53 -06:00
Suraj Gupta
a674edaa62 Address PR comments. 2021-12-10 16:43:58 -06:00
Suraj Gupta
310d990b12 Add debugging. 2021-12-10 16:43:58 -06:00
Suraj Gupta
968a4f9f50 Don't rely on database config to be updated. 2021-12-10 14:00:34 -06:00
Suraj Gupta
fdc94d3356 Fix blobGranuleVerifier setup. 2021-12-10 14:00:34 -06:00
Suraj Gupta
fc3376fe8f Move client knob to database config for blob granules. 2021-12-10 14:00:34 -06:00
Josh Slocum
b438e98220 Fixed multiple availability bugs in blob worker 2021-12-08 15:47:16 -06:00
Josh Slocum
6c79750412 Added a bunch of debugging for tracking whenAtLatest problems, and fixed a bug there 2021-12-08 12:41:45 -06:00
Josh Slocum
a82845af43 Merge branch 'master' into bg_bindings 2021-12-01 16:55:28 -06:00
sfc-gh-tclinkenbeard
07349869d9 Use fmt to address -Wformat warnings 2021-11-17 14:45:48 -08:00