33 Commits

Author SHA1 Message Date
Lukas Joswiak
73a7c32982
Add fdbcli command to read/write version epoch (#6480)
* Initialize cluster version at wall-clock time

Previously, new clusters would begin at version 0. After this change,
clusters will initialize at a version matching wall-clock time. Instead
of using the Unix epoch (or Windows epoch), FDB clusters will use a new
epoch, defaulting to January 1, 2010, 01:00:00+00:00. In the future,
this base epoch will be modifiable through fdbcli, allowing
administrators to advance the cluster version.

Basing the version off of time allows different FDB clusters to share
data without running into version issues.

* Send version epoch to master

* Cleanup

* Update fdbserver/storageserver.actor.cpp

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Jump directly to expected version if possible

* Fix initial version issue on storage servers

* Add random recovery offset to start version in simulation

* Type fixes

* Disable reference time by default

Enable on a cluster using the fdbcli command `versionepoch add 0`.

* Use correct recoveryTransactionVersion when recovering

* Allow version epoch to be adjusted forwards (to decrease the version)

* Set version epoch in simulation

* Add quiet database check to ensure small version offset

* Fix initial version issue on storage servers

* Disable reference time by default

Enable on a cluster using the fdbcli command `versionepoch add 0`.

* Add fdbcli command to read/write version epoch

* Cause recovery when version epoch is set

* Handle optional version epoch key

* Add ability to clear the version epoch

This causes version advancement to revert to the old methodology whereas
versions attempt to advance by about a million versions per second,
instead of trying to match the clock.

* Update transaction access

* Modify version epoch to use microseconds instead of seconds

* Modify fdbcli version target API

Move commands from `versionepoch` to `targetversion` top level command.

* Add fdbcli tests for

* Temporarily disable targetversion cli tests

* Fix version epoch fetch issue

* Fix Arena issue

* Reduce max version jump in simulation to 1,000,000

* Rework fdbcli API

It now requires two commands to fully switch a cluster to using the
version epoch. First, enable the version epoch with `versionepoch
enable` or `versionepoch set <versionepoch>`. At this point, versions
will be given out at a faster or slower rate in an attempt to reach the
expected version. Then, run `versionepoch commit` to perform a one time
jump to the expected version. This is essentially irreversible.

* Temporarily disable old targetversion tests

* Cleanup

* Move version epoch buggify to sequencer

This will cause some issues with the QuietDatabase check for the version
offset - namely, it won't do anything, since the version epoch is not
being written to the txnStateStore in simulation. This will get fixed in
the future.

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-04-08 12:33:19 -07:00
sfc-gh-tclinkenbeard
a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
A.J. Beamon
f8cddd3842 Add fdbcli support for tenants 2022-03-17 12:10:39 -07:00
Josh Slocum
773886515e Merge branch 'feature-range-feed' into blob_full_clean 2021-10-22 11:07:51 -05:00
Evan Tschannen
3f7df58a77 fixed a number of issues 2021-10-19 13:56:52 -07:00
Chaoguang Lin
a6940c6061 Change the return type of triggerddteaminfologCommandActor to bool to make it consistent with others 2021-09-21 10:06:04 -07:00
Chaoguang Lin
a871a9fac6 Refactor fileconfigure command 2021-09-21 10:06:04 -07:00
Chaoguang Lin
65956ae6b7 Refactor configure command; refactor changeConfig to template code to reuse existing tests 2021-09-21 10:06:04 -07:00
Chaoguang Lin
6cca6b9e6f Refactor exclude command 2021-09-14 11:04:02 -07:00
Chaoguang Lin
ff54ba8626 Refactor coordinators and include command 2021-09-14 11:04:02 -07:00
Chaoguang Lin
cb9dde358c Solve comments, refatctor the error handling part 2021-09-13 13:28:48 -07:00
Chaoguang Lin
cfb9b56067 Refactor lock/unlock command, fix the related correctness test, add the documentation for updating usage of the special key 2021-09-13 13:28:48 -07:00
Chaoguang Lin
af5643c998 Fix the issue in CheckStatus where it stuck as we cannot talk to any of the coordinators 2021-09-08 10:05:13 -07:00
Chaoguang Lin
7168594ae4 Refactor status command 2021-08-27 15:02:49 -07:00
Chaoguang Lin
46fd774d8c Move triggerddteaminfolog and tssq to new files; fix the reversed flag in tssq 2021-08-27 13:05:59 -07:00
Chaoguang Lin
1aa12faf01 Refactor profile command, remove profile heap|flow commands 2021-08-27 11:52:29 -07:00
Chaoguang Lin
09ccba67cf Refactor suspend and expensive_data_check commands 2021-08-25 12:05:51 -07:00
Chaoguang Lin
ce9bf5c5ce Create a new function to get all worker interfaces 2021-08-25 11:54:08 -07:00
Chaoguang Lin
41f09e9c2e Refactor kill command 2021-08-25 10:47:52 -07:00
Chaoguang Lin
0806309230
Merge pull request #5458 from sfc-gh-clin/refactor-transaction-related-commands
Refactor transaction related commands
2021-08-24 16:42:03 -07:00
Chaoguang Lin
8de3417f24 Fix comment, add a new function for reading special keys' error message 2021-08-24 14:18:43 -07:00
Chaoguang Lin
868fa93d3d Merge branch 'master' of github.com:apple/foundationdb into refactor-cache_range-command 2021-08-24 12:58:23 -07:00
Chaoguang Lin
a29e380516 Print out setclass special keys' related error if present 2021-08-24 11:51:12 -07:00
Chaoguang Lin
8c19969048 Refactor the cache_range command 2021-08-20 10:57:13 -07:00
Chaoguang Lin
815ad03122 Refactor datadistribution command 2021-08-06 18:27:45 +00:00
Chaoguang Lin
bbc2ffe527 Remove unnecessay header, add new line at the end of the file 2021-08-05 19:29:01 +00:00
Chaoguang Lin
7f7620d125 Refactor the setclass command 2021-08-05 19:01:17 +00:00
Chaoguang Lin
28128d79b1 Refactor throttle command 2021-07-27 17:58:11 +00:00
Chaoguang Lin
9b228fd4db Refactor snapshot command using the client function 2021-05-20 21:01:50 +00:00
Chaoguang Lin
f3854fe8da Refactor force_recovery_with_data_loss command using the corresponding client function 2021-05-20 19:47:00 +00:00
Chaoguang Lin
6df94cba98 Refactor maintenance command using special keys 2021-05-20 18:37:03 +00:00
Chaoguang Lin
35b88cd28b refactor advanceversion command 2021-05-18 07:22:17 +00:00
Chaoguang Lin
9a6151d3fc Update fdbcli.h to fdbcli.actor.h, removed the unnecessary wrapper 2021-05-11 14:31:08 -07:00