mirror of
https://github.com/apple/foundationdb.git
synced 2025-06-02 19:25:52 +08:00
* [fdbserver] workaround the FRT type layout issue to get Swfit getVersion working * MasterData.actor.h: fix comment typo * masterserver.swift: some tweaks * masterserver.swift: remove getVersion function, use the method * masterserver.swift: print replied version to output for tracing * [swift] add radar links for C++ interop issues found in getVersion bringup * Update fdbserver.actor.cpp * Migrate MasterData closer to full reference type This removes the workaround for the FRT type layout issue, and gets us closer to making MasterData a full reference type * [interop] require a new toolchain (>= Oct 19th) to build * [Swift] fix computation of toAdd for getVersion Swift implementation * add Swift to FDBClient and add async `atLeast` to NotifiedVersion * fix * use new atLeast API in master server * =build fixup link dependencies in swift fdbclient * clocks * +clock implement Clock using Flow's notion of time * [interop] workaround the immortal retain/release issue * [swift] add script to get latest centos toolchain * always install swift hooks; not only in "test" mode * simulator - first thing running WIP * cleanups * more cleanup * working snapshot * remove sim debug printlns * added convenience for whenAtLeast * try Alex's workaround * annotate nonnull * cleanup clock a little bit * fix missing impls after rebase * Undo the swift_lookup_Map_UID_CommitProxyVersionReplies workaround No longer needed - the issue was retain/release * [flow][swift] add Swift version of BUGGIFY * [swiftication] add CounterValue type to provide value semantics for Counter types on the Swift side * remove extraneous requestingProxyUID local * masterserver: initial Swift state prototype * [interop] make the Swiftied getVersion work * masterserver - remove the C++ implementation (it can't be supported as state is now missing) * Remove unnecessary SWIFT_CXX_REF_IMMORTAL annotations from Flow types * Remove C++ implementation of CommitProxyVersionReplies - it's in Swift now * [swift interop] remove more SWIFT_CXX_REF_IMMORTAL * [swift interop] add SWIFT_CXX_IMMORTAL_SINGLETON_TYPE annotation for semanticly meaningful immortal uses * rename SWIFT_CXX_REF_IMMORTAL -> UNSAFE_SWIFT_CXX_IMMORTAL_REF * Move master server waitForPrev to swift * =build fix linking swift in all modules * =build single link option * =cmake avoid manual math, just get "last" element from list * implement Streams support (#18) * [interop] update to new toolchain #6 * [interop] remove C++ vtable linking workarounds * [interop] make MasterData proper reference counted SWIFT_CXX_REF_MASTERDATA * [interop] use Swift array to pass UIDs to registerLastCommitProxyVersionReplies * [interop] expose MasterServer actor to C++ without wrapper struct * [interop] we no longer need expose on methods 🥳 * [interop] initial prototype of storing CheckedContinuation on the C++ side * Example of invoking a synchronous swift function from a C++ unit test. (#21) * move all "tests" we have in Swift, and priority support into real modules (#24) * Make set continuation functions inline * Split flow_swift into flow_swift and flow_swift_future to break circular dependency * rename SwiftContinuationCallbackStruct to FlowCallbackForSwiftContinuation * Future interop: use a method in a class template for continuation set call * Revert "Merge pull request #22 from FoundationDB/cpp-continuation" (#30) * Basic Swift Guide (#29) Co-authored-by: Alex Lorenz <arphaman@gmail.com> * Revert "Revert "Merge pull request #22 from FoundationDB/cpp-continuation" (#30)" This reverts commit c025fe6258c4c4904d5e70cd549d408bb61f2802. * Restore the C++ continuation, but it seems waitValue is broken for CInt somehow now * disable broken tests - waitValue not accessible * Streams can be async iterated over (#27) Co-authored-by: Alex Lorenz <arphaman@gmail.com> * remove work in progress things (#35) * remove some not used (yet) code * remove expose func for CInt, it's a primitive so we always have witness info (#37) * +masterdata implement provideVersions in Swift (#36) * serveLiveCommittedVersion in Swift (#38) * Port updateLiveCommittedVersion to swift (#33) Co-authored-by: Konrad `ktoso` Malawski <konrad_malawski@apple.com> * Implement updateRecoveryData in Swift (#39) Co-authored-by: Alex Lorenz <arphaman@gmail.com> * Simplify flow_swift to avoid multiple targets and generate separate CheckedContinuation header * Uncomment test which was blocked on extensions not being picked up (#31) * [interop] Use a separate target for Swift-to-C++ header generation * reduce boilerplate in future and stream support (#41) * [interop] require interop v8 - that will fix linker issue (https://github.com/apple/swift/issues/62448) * [interop] fix swift_stream_support.h Swift include * [interop] bump up requirement to version 9 * [interop] Generalize the Flow.Optional -> Swift.Optional conversion using generics * [WIP] masterServer func in Swift (#45) * [interop] Try conforms_to with a SWIFT_CONFORMS_TO macro for Optional conformance (#49) * [interop] include FlowOptionalProtocol source file when generating Flow_CheckedContinuation.h This header generation step depends on the import of the C++ Flow module, which requires the presence of FlowOptionalProtocol * conform Future to FlowFutureOps * some notes * move to value() so we can use discardable result for Flow.Void * make calling into Swift async funcs nicer by returning Flow Futures * [interop] hide initial use of FlowCheckedContinuation in flow.h to break dependency cycle * [fdbserver] fix an EncryptionOpsUtils.h modularization issue (showed up with modularized libc++) * Pass GCC toolchain using CMAKE_Swift_COMPILE_EXTERNAL_TOOLCHAIN to Swift's clang importer * [interop] drop the no longer needed libstdc++ include directories * [cmake] add a configuration check to ensure Swift can import C++ standard library * [swift] include msgpack from msgpack_DIR * [interop] make sure the FDB module maps have 'export' directive * add import 'flow_swift' to swift_fdbserver_cxx_swift_value_conformance.swift This is needed for CONFORMS_TO to work in imported modules * make sure the Swift -> C++ manually bridged function signature matches generated signature * [interop][workaround] force back use of @expose attribute before _Concurrency issue is fixed * [interop] make getResolutionBalancer return a pointer to allow Swift to use it We should revert back to a reference once compiler allows references again * [interop] add a workaround for 'pop' being marked as unsafe in Swift * masterserver.swift: MasterData returns the Swift actor pointer in an unsafe manner * Add a 'getCopy' method to AsyncVar to make it more Swift friendly * [interop] bump up the toolchain requirement * Revert "[interop][workaround] force back use of @expose attribute before _Concurrency issue is fixed" This reverts commit b01b271a76d1677bbb7c5c9c64cdad4b8b2b9612. * [interop] add FIXME comments highlighting new issue workarounds * [interop] adopt the new C++ interoperability compiler flag * [interop] generate swift compile commands * Do not deduplicate Swift compilation commands * [interop] generate swift compile commands * Do not deduplicate Swift compilation commands * flow actorcompiler.h: add a SWIFT_ACTOR empty macro definition This is needed to make the actor files parsable by clangd * [cmake] add missing dependencies * experimental cross compile * [cmake] fix triple in cross-compiled cmake flags * [interop] update to interop toolchain version 16 * [x-compile] add flags for cross-compiling boost * cleanup x-compile cmake changes * [cmake] fix typo in CMAKE_Swift_COMPILER_EXTERNAL_TOOLCHAIN config variable * [interop] pass MasterDataActor from Swift to C++ and back to Swift * [fdbserver] Swift->C++ header generation for FDBServer should use same module cache path * Update swift_get_latest_toolchain.sh to fetch 5.9 toochains * set HAVE_FLAG_SEARCH_PATHS_FIRST for cross compilation * Resolve conflicts in net2/sim2/actors, can't build yet * undo SWIFT_ACTOR changes, not necessary for merge * guard c++ compiler flags with is_cxx_compile * Update flow/actorcompiler/ActorParser.cs Co-authored-by: Evan Wilde <etceterawilde@gmail.com> * update the boost dependency * Include boost directory from the container for Swift * conform flow's Optional to FlowOptionalProtocol again * Guard entire RocksDBLogForwarder.h with SSD_ROCKSDB_EXPERIMENTAL to avoid failing on missing rocksdb APIs * remove extraneous merge marker * [swift] update swift_test_streams.swifto to use vars in more places * Add header guard to flow/include/flow/ThreadSafeQueue.h to fix moduralization issue * Update net and sim impls * [cmake] use prebuilt libc++ boost only when we're actually using libc++ * [fdbserver] Swift->C++ header generation for FDBServer should use same module cache path * fixups after merge * remove CustomStringConvertible conformance that would not be used * remove self-caused deprecation warnings in future_support * handle newly added task priority * reformatting * future: make value() not mutating * remove FIXME, not needed anymore * future: clarify why as functions * Support TraceEvent in Swift * Enable TraceEvent using a class wrapper in Swift * prearing WITH_SWIFT flag * wip disabled failing Go stuff * cleanup WITH_SWIFT_FLAG and reenable Go * wip disabled failing Go stuff * move setting flag before printing it * Add SWIFT_IDE_SETUP and cleanup guides and build a bit * Revert "Wipe packet buffers that held serialized WipedString (#10018)" This reverts commit e2df6e33029897360f8e11b3aea8fef97393a98c. * [Swift] Compile workaround in KeyBackedRangeMap; default init is incorrect * [interop] do not add FlowFutureOps conformance when building flow clang module for Flow checked continuation header pre-generation * make sure to show -DUSE_LIBCXX=OFF in readme * readme updates * do not print to stderr * Update Swift and C++ code to build with latest Swift 5.9 toolchain now that we no longer support universal references and bridge the methods that take in a constant reference template parameter correctly * Fix SERVER_KNOBS and enable use them for masterserver * Bump to C++20, Swift is now able to handle it as well * Put waitForPrev behind FLOW_WITH_SWIFT knob * Forward declare updateLiveCommittedVersion * Remove unused code * fix wrong condition set for updateLiveCommittedVersion * Revert "Revert "Wipe packet buffers that held serialized WipedString (#10018)"" This reverts commit 5ad8dce0525fb1844664ed2ccd7ba595db6913dd. * Enable go-bindings in cmake * Revert "Revert "Wipe packet buffers that held serialized WipedString (#10018)"" This reverts commit 5ad8dce0525fb1844664ed2ccd7ba595db6913dd. * USE_SWIFT flag so we "build without swift" until ready to by default * uncomment a few tests which were disabled during USE_SWIFT enablement * the option is WITH_SWIFT, not USE * formatting * Fix masterserver compile error * Fix some build errors. How did it not merge cleanly? :/ * remove initializer list from constructor * Expect Swift toolchain only if WITH_SWIFT is enabled * Don't require Flow_CheckedContinuation when Swift is disabled * Don't compile FlowCheckedContinuation when WITH_SWIFT=OFF * No-op Swift macros * More compile guards * fix typo * Run clang-format * Guard swift/bridging include in fdbrpc * Remove printf to pass the test * Remove some more printf to avoid potential issues TODO: Need to be TraceEvents instead * Remove __has_feature(nullability) as its only used in Swift * Don't use __FILENAME__ * Don't call generate_module_map outside WITH_SWIFT * Add some more cmake stuff under WITH_SWIFT guard * Some more guards * Bring back TLSTest.cpp * clang-format * fix comment formatting * Remove unused command line arg * fix cmake formatting in some files * Address some review comments * fix clang-format error --------- Co-authored-by: Alex Lorenz <arphaman@gmail.com> Co-authored-by: Russell Sears <russell_sears@apple.com> Co-authored-by: Evan Wilde <etceterawilde@gmail.com> Co-authored-by: Alex Lorenz <aleksei_lorenz@apple.com> Co-authored-by: Vishesh Yadav <vishesh_yadav@apple.com> Co-authored-by: Vishesh Yadav <vishesh3y@gmail.com>
271 lines
5.8 KiB
C++
271 lines
5.8 KiB
C++
/*
|
|
* swift_concurrency_hooks.cpp
|
|
*
|
|
* This source file is part of the FoundationDB open source project
|
|
*
|
|
* Copyright 2013-2022 Apple Inc. and the FoundationDB project authors
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License");
|
|
* you may not use this file except in compliance with the License.
|
|
* You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software
|
|
* distributed under the License is distributed on an "AS IS" BASIS,
|
|
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
* See the License for the specific language governing permissions and
|
|
* limitations under the License.
|
|
*/
|
|
|
|
#include "flow/swift.h"
|
|
#include "flow/swift_concurrency_hooks.h"
|
|
#include "flow/swift/ABI/Task.h"
|
|
#include "flow/TLSConfig.actor.h"
|
|
|
|
// FIXME: surely there must be some more automatic way to maintain the mappings with Swift/C++ interop.
|
|
TaskPriority swift_priority_to_net2(swift::JobPriority p) {
|
|
// printf("[c++][%s:%d](%s) converting a priority (priority: %zu)\n", __FILE_NAME__, __LINE__, __FUNCTION__, p);
|
|
|
|
TaskPriority fp = TaskPriority::Zero;
|
|
switch (static_cast<std::underlying_type<swift::JobPriority>::type>(p)) {
|
|
case 255:
|
|
fp = TaskPriority::Max;
|
|
break;
|
|
case 200:
|
|
fp = TaskPriority::RunLoop;
|
|
break;
|
|
case 173:
|
|
fp = TaskPriority::ASIOReactor;
|
|
break;
|
|
case 73:
|
|
fp = TaskPriority::RunCycleFunction;
|
|
break;
|
|
case 72:
|
|
fp = TaskPriority::FlushTrace;
|
|
break;
|
|
case 71:
|
|
fp = TaskPriority::WriteSocket;
|
|
break;
|
|
case 70:
|
|
fp = TaskPriority::PollEIO;
|
|
break;
|
|
case 69:
|
|
fp = TaskPriority::DiskIOComplete;
|
|
break;
|
|
case 68:
|
|
fp = TaskPriority::LoadBalancedEndpoint;
|
|
break;
|
|
case 67:
|
|
fp = TaskPriority::ReadSocket;
|
|
break;
|
|
case 66:
|
|
fp = TaskPriority::AcceptSocket;
|
|
break;
|
|
case 65:
|
|
fp = TaskPriority::Handshake;
|
|
break;
|
|
case 64:
|
|
fp = TaskPriority::CoordinationReply;
|
|
break;
|
|
case 63:
|
|
fp = TaskPriority::Coordination;
|
|
break;
|
|
case 62:
|
|
fp = TaskPriority::FailureMonitor;
|
|
break;
|
|
case 61:
|
|
fp = TaskPriority::ResolutionMetrics;
|
|
break;
|
|
case 60:
|
|
fp = TaskPriority::Worker;
|
|
break;
|
|
case 59:
|
|
fp = TaskPriority::ClusterControllerWorker;
|
|
break;
|
|
case 58:
|
|
fp = TaskPriority::ClusterControllerRecruit;
|
|
break;
|
|
case 57:
|
|
fp = TaskPriority::ClusterControllerRegister;
|
|
break;
|
|
case 56:
|
|
fp = TaskPriority::ClusterController;
|
|
break;
|
|
case 55:
|
|
fp = TaskPriority::MasterTLogRejoin;
|
|
break;
|
|
case 54:
|
|
fp = TaskPriority::ProxyStorageRejoin;
|
|
break;
|
|
case 53:
|
|
fp = TaskPriority::TLogQueuingMetrics;
|
|
break;
|
|
case 52:
|
|
fp = TaskPriority::TLogPop;
|
|
break;
|
|
case 51:
|
|
fp = TaskPriority::TLogPeekReply;
|
|
break;
|
|
case 50:
|
|
fp = TaskPriority::TLogPeek;
|
|
break;
|
|
case 49:
|
|
fp = TaskPriority::TLogCommitReply;
|
|
break;
|
|
case 48:
|
|
fp = TaskPriority::TLogCommit;
|
|
break;
|
|
case 47:
|
|
fp = TaskPriority::ReportLiveCommittedVersion;
|
|
break;
|
|
case 46:
|
|
fp = TaskPriority::ProxyGetRawCommittedVersion;
|
|
break;
|
|
case 45:
|
|
fp = TaskPriority::ProxyMasterVersionReply;
|
|
break;
|
|
case 44:
|
|
fp = TaskPriority::ProxyCommitYield2;
|
|
break;
|
|
case 43:
|
|
fp = TaskPriority::ProxyTLogCommitReply;
|
|
break;
|
|
case 42:
|
|
fp = TaskPriority::ProxyCommitYield1;
|
|
break;
|
|
case 41:
|
|
fp = TaskPriority::ProxyResolverReply;
|
|
break;
|
|
case 40:
|
|
fp = TaskPriority::ProxyCommit;
|
|
break;
|
|
case 39:
|
|
fp = TaskPriority::ProxyCommitBatcher;
|
|
break;
|
|
case 38:
|
|
fp = TaskPriority::TLogConfirmRunningReply;
|
|
break;
|
|
case 37:
|
|
fp = TaskPriority::TLogConfirmRunning;
|
|
break;
|
|
case 36:
|
|
fp = TaskPriority::ProxyGRVTimer;
|
|
break;
|
|
case 35:
|
|
fp = TaskPriority::GetConsistentReadVersion;
|
|
break;
|
|
case 34:
|
|
fp = TaskPriority::GetLiveCommittedVersionReply;
|
|
break;
|
|
case 33:
|
|
fp = TaskPriority::GetLiveCommittedVersion;
|
|
break;
|
|
case 32:
|
|
fp = TaskPriority::GetTLogPrevCommitVersion;
|
|
break;
|
|
case 31:
|
|
fp = TaskPriority::UpdateRecoveryTransactionVersion;
|
|
break;
|
|
case 30:
|
|
fp = TaskPriority::DefaultPromiseEndpoint;
|
|
break;
|
|
case 29:
|
|
fp = TaskPriority::DefaultOnMainThread;
|
|
break;
|
|
case 28:
|
|
fp = TaskPriority::DefaultDelay;
|
|
break;
|
|
case 27:
|
|
fp = TaskPriority::DefaultYield;
|
|
break;
|
|
case 26:
|
|
fp = TaskPriority::DiskRead;
|
|
break;
|
|
case 25:
|
|
fp = TaskPriority::DefaultEndpoint;
|
|
break;
|
|
case 24:
|
|
fp = TaskPriority::UnknownEndpoint;
|
|
break;
|
|
case 23:
|
|
fp = TaskPriority::MoveKeys;
|
|
break;
|
|
case 22:
|
|
fp = TaskPriority::DataDistributionLaunch;
|
|
break;
|
|
case 21:
|
|
fp = TaskPriority::Ratekeeper;
|
|
break;
|
|
case 20:
|
|
fp = TaskPriority::DataDistribution;
|
|
break;
|
|
case 19:
|
|
fp = TaskPriority::DataDistributionLow;
|
|
break;
|
|
case 18:
|
|
fp = TaskPriority::DataDistributionVeryLow;
|
|
break;
|
|
case 17:
|
|
fp = TaskPriority::BlobManager;
|
|
break;
|
|
case 16:
|
|
fp = TaskPriority::DiskWrite;
|
|
break;
|
|
case 15:
|
|
fp = TaskPriority::UpdateStorage;
|
|
break;
|
|
case 14:
|
|
fp = TaskPriority::CompactCache;
|
|
break;
|
|
case 13:
|
|
fp = TaskPriority::TLogSpilledPeekReply;
|
|
break;
|
|
case 12:
|
|
fp = TaskPriority::BlobWorkerReadChangeFeed;
|
|
break;
|
|
case 11:
|
|
fp = TaskPriority::BlobWorkerUpdateFDB;
|
|
break;
|
|
case 10:
|
|
fp = TaskPriority::BlobWorkerUpdateStorage;
|
|
break;
|
|
case 9:
|
|
fp = TaskPriority::FetchKeys;
|
|
break;
|
|
case 8:
|
|
fp = TaskPriority::RestoreApplierWriteDB;
|
|
break;
|
|
case 7:
|
|
fp = TaskPriority::RestoreApplierReceiveMutations;
|
|
break;
|
|
case 6:
|
|
fp = TaskPriority::RestoreLoaderFinishVersionBatch;
|
|
break;
|
|
case 5:
|
|
fp = TaskPriority::RestoreLoaderSendMutations;
|
|
break;
|
|
case 4:
|
|
fp = TaskPriority::RestoreLoaderLoadFiles;
|
|
break;
|
|
case 3:
|
|
fp = TaskPriority::LowPriorityRead;
|
|
break;
|
|
case 2:
|
|
fp = TaskPriority::Low;
|
|
break;
|
|
case 1:
|
|
fp = TaskPriority::Min;
|
|
break;
|
|
case 0:
|
|
fp = TaskPriority::Zero;
|
|
break;
|
|
default: {
|
|
printf("[c++]Unknown priority: %zu\n", p);
|
|
abort();
|
|
}
|
|
}
|
|
// return static_cast<std::underlying_type<TaskPriority>::type>(fp);
|
|
return fp;
|
|
}
|