Jingyu Zhou
1a5bf25b5c
Update code base to use fmt 8.1.1
2022-03-04 15:52:06 -08:00
A.J. Beamon
250a88e682
Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement.
2022-02-24 12:25:52 -08:00
Evan Tschannen
b11ae4dae8
Merge pull request #5910 from sfc-gh-jslocum/bg_bindings
...
Blob Granule C bindings
2021-12-02 11:40:26 -08:00
sfc-gh-tclinkenbeard
90ced244eb
Fix -Wunused-but-set-variable warnings
2021-12-01 18:15:53 -08:00
Josh Slocum
a82845af43
Merge branch 'master' into bg_bindings
2021-12-01 16:55:28 -06:00
sfc-gh-tclinkenbeard
2613ec7561
Expand use of fmt to get rid of %ld usage
2021-11-17 17:03:32 -08:00
sfc-gh-tclinkenbeard
07349869d9
Use fmt to address -Wformat warnings
2021-11-17 14:45:48 -08:00
sfc-gh-tclinkenbeard
766a05d33c
Merge remote-tracking branch 'origin/master' into add-format-warning
2021-11-17 12:14:01 -08:00
Evan Tschannen
6909754b21
changefeeds now have a whenAtLeast function for efficiently learning when the version has updated but no mutations have been committed
2021-11-14 19:08:46 -08:00
Josh Slocum
5b2617a524
Added local granule file reading to mako
2021-11-03 09:33:30 -05:00
Josh Slocum
382882f1c1
mako successfully calls read_blob_granules and gets stuff back
2021-11-02 13:43:42 -05:00
sfc-gh-tclinkenbeard
13bb7838aa
Enable clang -Wformat warning
2021-10-30 21:07:38 -07:00
Josh Slocum
773886515e
Merge branch 'feature-range-feed' into blob_full_clean
2021-10-22 11:07:51 -05:00
Josh Slocum
912ef76f1c
cleanup before merge
2021-10-18 17:11:14 -05:00
Suraj Gupta
41e96b98b2
Rethrow on actor cancelled.
2021-10-18 18:06:15 -04:00
Suraj Gupta
b15e4f6b4e
Fix reference cycle between BWData and GranuleRangeMetadata.
2021-10-18 17:03:27 -04:00
Suraj Gupta
e2e852e515
Mitigate transitive includes.
2021-10-18 10:49:25 -04:00
Suraj Gupta
28c0cd3fc6
Remove old comments from BW.
2021-10-18 09:56:01 -04:00
Josh Slocum
8dc5ae9d24
fixed version boundaries
2021-10-14 10:45:09 -05:00
Josh Slocum
b5074fd597
Reworked all of the system data to encode granule data more efficiently for persistence
2021-10-13 16:28:04 -05:00
Josh Slocum
f3c44c568f
fixing merge conflicts
2021-10-13 16:26:44 -05:00
Suraj Gupta
cfb8368da6
Address PR comments.
2021-10-13 14:56:17 -04:00
Suraj Gupta
d002df3b21
Clean up blob worker changes.
2021-10-13 14:40:26 -04:00
Suraj Gupta
266a5b06fa
Fix infinite loop.
2021-10-13 14:40:26 -04:00
Suraj Gupta
423a67f448
trying to fix infinite loop
2021-10-13 14:40:26 -04:00
Suraj Gupta
dfb9655c57
Handle blob work failure
2021-10-13 14:40:26 -04:00
Josh Slocum
4e4a2534da
review comments and moving priority yield to correct spot
2021-10-06 09:24:48 -05:00
Josh Slocum
6a24ef9258
adding priorities to blob worker and fixing monitoring in blob manager
2021-10-05 16:51:19 -05:00
Suraj Gupta
282f9d35cd
Cleanup comments and debugging code.
2021-10-04 11:07:08 -04:00
Suraj Gupta
4d54669ccd
Recruit the blob workers via blob manager.
...
In this PR, the blob manager now recruits blob workers
(via communication with the cluster controller). Blob workers
are onboarded as blob worker processes enter the cluster.
2021-10-04 11:07:08 -04:00
Josh Slocum
8fb7c45e65
even more bug fixes
2021-09-25 10:30:27 -05:00
Josh Slocum
3b23d6aba5
rollback fixes
2021-09-25 09:19:00 -05:00
Josh Slocum
fa1fe5f08b
added blob worker rollbacks that handle (most) cases
2021-09-24 17:52:36 -05:00
Josh Slocum
a986679b7f
More bug fixes
2021-09-24 10:02:02 -05:00
Josh Slocum
5ddf08dfe5
Got basic range reassignment working
2021-09-22 16:48:44 -05:00
Josh Slocum
8e4582673c
more bug fixes
2021-09-20 10:21:52 -05:00
Josh Slocum
9a92b763ae
Fixing issues in known committed version tracking
2021-09-17 12:26:10 -05:00
Josh Slocum
c9b8bdffbe
Passes a single correctness test!
2021-09-15 17:18:04 -05:00
Josh Slocum
cefb66d64c
fixing a couple bugs
2021-09-10 12:52:33 -05:00
Josh Slocum
e2a51a4fe7
Fixing up after change feed updates
2021-09-10 11:49:41 -05:00
Josh Slocum
44a64005cf
also pipelining snapshot from blob in blob worker for maximum throughput
2021-09-09 10:43:28 -05:00
Josh Slocum
99e5ecdf9d
Doing GRVs for committed version checking in blob worker
2021-09-08 17:24:49 -05:00
Josh Slocum
2491287f98
Pipelined delta file writing to improve performance
2021-09-08 16:57:57 -05:00
Josh Slocum
eb76343dfb
Added blob granule reassignment and splitting
2021-09-08 14:09:14 -05:00
Suraj Gupta
5aebd77f0a
PR changes
2021-09-03 13:58:39 -04:00
Suraj Gupta
a0b3446572
Add metrics for blob worker.
...
We want to add metrics for the blob worker to evaluate its
performance more concretely. We decided to track the following
information:
- s3 put requests
- s3 get requests
- S3 delete requests
- Delta files written
- Snapshot files written
- Delta bytes written
- Snapshot bytes written
- Number of current ranges assigned
- Bytes read from FDB (initial snapshot)
- Bytes read from S3 (compaction)
- Read requests count
- Read files returned
- Read deltas returned
- Read delta bytes returned
- Ranges assigned
- Ranges revoked
- Number of current ranges assigned
- Total mutation bytes buffered across all ranges // current or accumulated
- Range feed bytes input
- Range feed mutations input
- Wrong Shard Server count
2021-09-03 12:09:01 -04:00
Suraj Gupta
bdbb0303f3
Add missing wait around deleteFile invocations.
2021-09-03 11:12:03 -04:00
Josh Slocum
ff96843c58
review comments
2021-09-01 10:12:53 -05:00
Josh Slocum
46adada5ff
Cleaned up debugging and fixed a couple bugs
2021-08-31 12:30:43 -05:00
Josh Slocum
b4bfd58bcb
multiple blob workers appears to work
2021-08-30 13:07:25 -05:00
