95 Commits

Author SHA1 Message Date
Markus Pilman
09c136e434 Framework for transaction tracing 2020-06-08 16:09:37 -07:00
negoyal
23a565ec63 Few bug fixes. 2020-06-05 16:27:04 -07:00
negoyal
cf13e00a8f Merge remote-tracking branch 'origin/release-6.3' into fdb_cache_wo_allocator 2020-06-01 17:38:31 -07:00
Markus Pilman
ba63a66d12 Merge remote-tracking branch 'origin/master' into features/flatbuffers-debugtx 2020-05-29 15:38:09 -07:00
negoyal
2e93e09da5 Fix the lock. 2020-05-29 13:07:38 -07:00
Markus Pilman
8a7d98ab21 Started instrumentation 2020-05-27 15:36:04 -07:00
A.J. Beamon
7a09d016a6 Merge branch 'release-6.3' into merge-release6.3-into-master 2020-05-19 12:52:44 -07:00
Markus Pilman
d748166ed3 Merge branch 'master' of github.com:apple/foundationdb into features/flatbuffers-debugtx 2020-05-18 10:15:58 -07:00
Alex Miller
ccaac162e2 Resolve performance concerns of nearly-no-op debugMutation being frequently called
This introduces unhygenic macro variants that inline a `ENABLED &&`
before the TraceEvent.  This way, they get entirely compiled out unless
enabled.

Then rewrite all debugMutation uses via sed.
2020-05-13 18:44:15 -07:00
Markus Pilman
5f9b127e56 Emit traces regularly about role assignment
We are currently emitting Role transition traces when a role starts and
when it ends. While this is useful for debugging, it doesn't work well
with tools that inject data and might potentially miss some trace lines.

We do decorate each trace lines with the roles assigned to that
particular process, however, this is not sufficient for tools that can
make use of the UID -> Role mapping
2020-05-08 16:27:57 -07:00
negoyal
749fcd13b0 Merge branch 'master' into fdb_cache_wo_allocator 2020-05-08 16:23:29 -07:00
negoyal
049e3d36be Revert custom allocator. 2020-05-08 15:43:48 -07:00
negoyal
90cf28d341 Fix a version related issue. 2020-05-07 12:33:14 -07:00
negoyal
116e186af6 Add code to fetch cached ranges when a cache server startsup. 2020-05-06 18:56:42 -07:00
negoyal
dd033736ed Merge branch 'master' into fdb_cache_subfeature2 2020-05-04 17:29:43 -07:00
Markus Pilman
451eb041fb Made interface nicer 2020-04-22 14:50:33 -07:00
Markus Pilman
e22b88d857 Revert "Add FBTrace calls for most debugTransaction trace messages."
This reverts commit be68bd6ad239be0a6cc47ee42f85e929d89310a1.
2020-04-16 13:14:27 -07:00
negoyal
9034964864 Compilation error fix. 2020-04-14 18:21:43 -07:00
negoyal
7a9bcf8222 Review comments. 2020-04-14 17:45:34 -07:00
Neelam Goyal
ec4512be2b
Update fdbserver/StorageCache.actor.cpp
Co-Authored-By: Trevor Clinkenbeard <trevorclinkenbeard@gmail.com>
2020-04-13 12:27:30 -07:00
Neelam Goyal
b80df3e166
Update fdbserver/StorageCache.actor.cpp
Co-Authored-By: Trevor Clinkenbeard <trevorclinkenbeard@gmail.com>
2020-04-13 12:24:32 -07:00
Neelam Goyal
2f8e5e7943
Update fdbserver/StorageCache.actor.cpp
Co-Authored-By: Trevor Clinkenbeard <trevorclinkenbeard@gmail.com>
2020-04-13 12:00:30 -07:00
negoyal
be68bd6ad2 Add FBTrace calls for most debugTransaction trace messages. 2020-04-09 16:45:24 -07:00
negoyal
e24017fae3 Minor definition fixes. 2020-04-06 17:53:26 -07:00
negoyal
740c5490b3 Added flat buffer classes for debugTransaction trace message types. 2020-04-06 16:00:40 -07:00
Alex Miller
72e5891058 Clean up and rework the debugMutation API.
As a relatively unknown debugging tool for simulation tests, one could
have simulation print when a particular key is handled in various stages
of the commit process.  This functionality was enabled by changing a 0
to a 1 in an #if, and changing a constant to the key in question.

As a proxy and storage server handle mutations, they call debugMutation
or debugKeyRange, which then checks against the mutation against the key
in question, and logs if they match.  A mixture of printfs and
TraceEvents would then be emitted, and for this to actually be usable,
one also needs to comment out some particularly spammy debugKeyRange()
calls.

This PR reworks the API of debugMutation/debugKeyRange, pulls it out
into its own file, and trims what is logged by default into something
useful and understandable:
* debugMutation() now returns a TraceEvent, that one can add more details to before it is logged.
* Data distribution and storage server cleanup operations are no longer logged by default
2020-03-27 03:30:28 -07:00
negoyal
8abac91033 Fixed a bug in cache server while peeking at a version lower than popped version and added some logging. 2020-03-26 12:39:07 -07:00
negoyal
99a5cb0572 Fix a corruption bug. 2020-03-13 18:52:34 -07:00
negoyal
975340cb4d Convert the future_version SevError to SevDebug in cache role. 2020-03-04 15:18:49 -08:00
negoyal
3acd3ad3af Some bugfixes and cleanup. 2020-03-02 17:11:23 -08:00
negoyal
0cbd253967 Reduce the amount of tracing in cache server. 2020-02-25 17:32:24 -08:00
negoyal
308e088bca Minor fixes. 2020-02-25 15:00:18 -08:00
negoyal
7193fa87b8 And some more compilation fixes. 2020-02-25 10:45:02 -08:00
negoyal
a21f29b586 Bugfixes. Identify the cached range correctly. 2020-02-19 16:27:06 -08:00
negoyal
28768715f2 Custom bookkeeping allocator for PTree. Simple Cache workload. 2020-02-14 11:42:47 -08:00
Andrew Noyes
09f3690f09 Fix OPEN_FOR_IDE build 2020-02-03 10:42:05 -08:00
negoyal
b82c3be116 Throw future_version error for cold caches. 2020-01-14 14:29:41 -08:00
negoyal
34adf9c92c Storages now return a bool flag to indicate if the key[s] might be cached. 2020-01-14 12:33:17 -08:00
Markus Pilman
4fa619e782
Merge branch 'fdb_cache_subfeature2' into features/cache-initialization 2020-01-07 19:59:05 -08:00
mpilman
d3d6016c90 Merge remote-tracking branch 'negoyal/fdb_cache_subfeature2' into features/cache-initialization 2020-01-07 19:53:09 -08:00
negoyal
e8e5f2d118 Bug fixes in the cache role. 2020-01-07 16:51:40 -08:00
negoyal
29b77863f0 Cache warmup and Consistency check workload changes. 2020-01-07 13:06:58 -08:00
mpilman
821edcb207 Register caches through keyspace
This also removes the old mechanism that registers them
through the serverDBInfo.

Caches do now self-recruit at startup
2019-12-06 13:28:44 -08:00
negoyal
cf2563f1c7 Mix of various things, a lot of which will change. 2019-12-05 17:10:32 -08:00
negoyal
a4a0bf18f9 Merging with Master. 2019-11-12 13:01:29 -08:00