67 Commits

Author SHA1 Message Date
Ray Jenkins
1c5bf135d5
Revert "Migrate to OpenTelemetry tracing. (#6855)" (#6941)
This reverts commit 5df3bac110d9b5b88931b008b852433688bb7eb0.
2022-04-25 09:29:56 -05:00
Ray Jenkins
5df3bac110
Migrate to OpenTelemetry tracing. (#6855) 2022-04-20 09:26:37 -05:00
Jingyu Zhou
7cd5ef711d Fix test failure to BlobGranule due to missing private mutations
The change feed metadata mutations use \xff\x02/feed/ prefix, which was not
considered as "metadata mutations", thus not sent to the resolvers. This makes
the private mutation generation not possible for change feed if the knob
PROXY_USE_RESOLVER_PRIVATE_MUTATIONS is on. Fix by making it a metadata
mutation.
2022-04-01 14:23:03 -07:00
Jingyu Zhou
cfcf0f152c Merge branch 'main-4a085fc84' into vv
Fix Conflicts:
	fdbclient/NativeAPI.actor.cpp
	fdbserver/ClusterRecovery.actor.cpp
	fdbserver/MasterInterface.h
	fdbserver/masterserver.actor.cpp
	flow/error_definitions.h
2022-03-30 22:28:06 -07:00
Jingyu Zhou
00b57d4cce Merge branch 'main-67eba5ec7' into vv
Fix Conflicts:
	fdbclient/CommitProxyInterface.h
	fdbclient/NativeAPI.actor.cpp
	fdbclient/StorageServerInterface.h
	fdbserver/CommitProxyServer.actor.cpp
	fdbserver/storageserver.actor.cpp
2022-03-30 20:05:55 -07:00
sfc-gh-tclinkenbeard
a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
A.J. Beamon
5fa9d3e1b7 Add a tenant parameter to read and commit requests. Store a map of all tenants on commit proxy and storage servers. Add an option to require tenant mode. 2022-03-06 21:54:21 -08:00
Jingyu Zhou
fc7325e923 Fix force recovery when generating private mutations
Resolver need to skip transactions after forceRecovery flag is true.
2021-10-15 09:47:23 -04:00
Jingyu Zhou
0dc9c607f4 Add knob PROXY_USE_RESOLVER_PRIVATE_MUTATIONS
To control proxy to use private mutations from resolvers or not.
2021-10-15 09:47:23 -04:00
Jingyu Zhou
90a27bedf5 fix conflict fdbserver/ApplyMetadataMutation.cpp 2021-10-15 09:47:20 -04:00
Jingyu Zhou
ff75d22ce8 Fix test failures
Need to be careful what values applyMetadataMutation needs.

Need to call txnStateStore->enableSnapshot() after receiving the broadcast to
allow semiCommit to happen.
2021-10-15 09:45:57 -04:00
Jingyu Zhou
fbc6f45190 Generate private mutations at Resolvers 2021-10-15 09:45:57 -04:00
Jingyu Zhou
e0184bc8aa resolve conflict 2021-10-15 09:45:52 -04:00
Dan Lambright
f3ebc47e75 Respond to comments 9/27 2021-09-30 11:51:57 -04:00
Dan Lambright
58e1888d8e remove network hop by getting previous commit versions in GetCommitVersionRequest 2021-09-30 11:51:57 -04:00
Xiaoge Su
a2114550e0 Refactor ApplyMetadataMutation for better readability 2021-08-24 12:58:14 -07:00
FDB Formatster
df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Trevor Clinkenbeard
24ea35e56f
Merge pull request #3748 from sfc-gh-ljoswiak/visibility-2
Add TLogVersion::V6
2020-10-14 17:35:32 -07:00
sfc-gh-tclinkenbeard
cf5337daa0 Improve performance of metadata check in applyMetadataMutations 2020-10-10 14:45:04 -07:00
sfc-gh-tclinkenbeard
dbeb8772f0 Fix isMetadataMutation 2020-09-19 09:40:40 -07:00
Lukas Joswiak
2a58e775d2 Add original changes 2020-09-04 15:36:47 -07:00
Xiaoge Su
d8bb36c4c8 Refactor applyMetadataMutations to accept less args 2020-08-06 11:40:40 -07:00
mpilman
d09e07f1f5 Merge remote-tracking branch 'upstream/master' into features/icc 2020-02-04 10:26:18 -08:00
negoyal
a4a0bf18f9 Merging with Master. 2019-11-12 13:01:29 -08:00
Alex Miller
a79c5d8200 Move code in ApplyMetadataMutation from .h to .cpp
Not only can we now compile this giant code block in parallel with the
rest of the code, we also no longer compile this giant function twice!
(Once for the master, once for the proxy.)
2019-07-16 15:00:34 -07:00
mpilman
844dd60202 FDB compiling with intel compiler 2019-06-20 09:29:01 -07:00
Evan Tschannen
b451c2cd56
Merge pull request #1497 from alexmiller-apple/fastrecovery
Add an \xff keyrange that is backed by the txnStateStore.
2019-05-23 10:52:35 -07:00
Alex Miller
f367385a80 Add clearing 2019-04-29 15:10:52 -07:00
Alex Miller
797d431934 Add an \xff keyrange that is backed by the txnStateStore. 2019-04-25 17:04:20 -07:00
Jingyu Zhou
966ec30fcc Add pseudoLocalities for special tag consumers 2019-04-21 10:41:07 -07:00
Jingyu Zhou
b4e7e7a85b Refactor StorageCache updates 2019-04-21 10:41:07 -07:00
mpilman
39ecbedd74 Fixed compilation errors on OS X & gcc8 2019-04-05 13:12:19 -07:00
mpilman
1c16f87a4e Remove trace-calls to printable (in non-workloads) 2019-04-05 13:12:19 -07:00
Evan Tschannen
3d196c9e97 fix: metadataVersionKey log range removal needs to be checked for each logDestination 2019-03-04 20:56:31 -08:00
Evan Tschannen
3da85f3acd implemented the \xff/metadataVersion key, which can be used by layers to help them cheaply cache metadata and know when their cache is invalid 2019-02-28 17:45:00 -08:00
Evan Tschannen
b8910ba7cd Merge branch 'master' into feature-fix-force-recovery
# Conflicts:
#	fdbclient/ManagementAPI.actor.h
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/KillRegion.actor.cpp
2019-02-22 14:38:13 -08:00
Evan Tschannen
3a572b010f fix: a forced recovery needed to force the data distributor to restart 2019-02-19 16:04:52 -08:00
mpilman
78dd80ea8a Proper fwd decl in BackupAgent
Also BackupAgent.h -> BackupAgent.actor.h
2019-02-19 15:16:59 -08:00
Evan Tschannen
065a45e05f Merge branch 'master' into feature-fix-force-recovery
# Conflicts:
#	fdbclient/ManagementAPI.actor.cpp
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/workloads/KillRegion.actor.cpp
2019-02-18 17:09:06 -08:00
Evan Tschannen
05ca0a10d8 fix: kill all storage servers which are not in the safe locality after a forced recovery 2019-02-18 14:30:51 -08:00
Jingyu Zhou
62c67a50e5 Fix segfault error
The usedIds is updated by master registration request, which populates the
usedIds map. However, this request may contain processes that cluster controller
is not aware, i.e., not in id_worker map.

This is ok until I added tracing the usedIds, which silently insert an empty
entry into id_worker map for the unknown process. This new entry can cause
crashing failure when trying to access its LocalityData.

Remove AsyncTrigger for usedIds, and change to serverInfo->onChange.

Use const & to avoid unnecessary copies in WorkerInterface's LocalityData
and getExtraTLogEligibleMachines().
2019-02-14 16:37:16 -08:00
A.J. Beamon
d4d5740282 * Add Optional.map and ErrorOr.map.
* Rename Optional/ErrorOr cast_to to castTo.
* Make printable(Optional<T>) templated rather than restricted to StringRef types.
* Fixes bug in (unused) ErrorOr.castTo where an ErrorOr that was not set would lose its error.
2019-01-11 09:03:38 -08:00
Robert Escriva
268093a96d Adjust all includes to be relative to the root.
Remove the use of relative paths.  A header at foo/bar.h could be included by
files under foo/ with "bar.h", but would be included everywhere else as
"foo/bar.h".  Adjust so that every include references such a header with the
latter form.

Signed-off-by: Robert Escriva <rescriva@dropbox.com>
2018-10-19 17:35:33 +00:00
Evan Tschannen
ffde1a0e28 renamed onlySystem to mustContainSystemMutations, to accurately represent what setting the key does 2018-08-21 22:15:45 -07:00
Evan Tschannen
cb60002944 Added the ability to disable all commits which do not modify the system keys by setting \xff/onlySystem = 1 in the database 2018-08-21 21:09:50 -07:00
A.J. Beamon
e5488419cc Attempt to normalize trace events:
* Detail names now all start with an uppercase character and contain no underscores. Ideally these should be head-first camel case, though that was harder to check.
* Type names have the same rules, except they allow one underscore (to support a usage pattern Context_Type). The first character after the underscore is also uppercase.
* Use seconds instead of milliseconds in details.

Added a check when events are logged in simulation that logs a message to stderr if the first two rules above aren't followed.

This probably doesn't address every instance of the above problems, but all of the events I was able to hit in simulation pass the check.
2018-06-08 11:11:08 -07:00
Evan Tschannen
fa9089c2e8 fix: removed storage servers must be popped on remote logs from the proxy 2018-04-25 15:38:34 -07:00
Evan Tschannen
37a6a81634 Merge commit '7f6fc3e039c911cd84b8540f7f799fc38a1c1822' into feature-remote-logs
# Conflicts:
#	fdbserver/workloads/RestartRecovery.actor.cpp
2018-02-23 12:33:28 -08:00
Alec Grieser
0bae9880f1 remove trailing whitespace from our copyright headers ; fixed formatting of python setup.py 2018-02-21 10:25:11 -08:00
Evan Tschannen
766964ff48 fix: dest tags were not repopulated when the tag cache was cleared 2018-01-31 17:35:48 -08:00