foundationdb/flow/clock_support.swift
Konrad `ktoso` Malawski c26aa0b2a3
Introduce initial Swift support in fdbserver (#10156)
* [fdbserver] workaround the FRT type layout issue to get Swfit getVersion working

* MasterData.actor.h: fix comment typo

* masterserver.swift: some tweaks

* masterserver.swift: remove getVersion function, use the method

* masterserver.swift: print replied version to output for tracing

* [swift] add radar links for C++ interop issues found in getVersion bringup

* Update fdbserver.actor.cpp

* Migrate MasterData closer to full reference type

This removes the workaround for the FRT type layout issue, and gets us closer to making MasterData a full reference type

* [interop] require a new toolchain (>= Oct 19th) to build

* [Swift] fix computation of toAdd for getVersion Swift implementation

* add Swift to FDBClient and add async `atLeast` to NotifiedVersion

* fix

* use new atLeast API in master server

* =build fixup link dependencies in swift fdbclient

* clocks

* +clock implement Clock using Flow's notion of time

* [interop] workaround the immortal retain/release issue

* [swift] add script to get latest centos toolchain

* always install swift hooks; not only in "test" mode

* simulator - first thing running WIP

* cleanups

* more cleanup

* working snapshot

* remove sim debug printlns

* added convenience for whenAtLeast

* try Alex's workaround

* annotate nonnull

* cleanup clock a little bit

* fix missing impls after rebase

* Undo the swift_lookup_Map_UID_CommitProxyVersionReplies workaround

No longer needed - the issue was retain/release

* [flow][swift] add Swift version of BUGGIFY

* [swiftication] add CounterValue type to provide value semantics for Counter types on the Swift side

* remove extraneous requestingProxyUID local

* masterserver: initial Swift state prototype

* [interop] make the Swiftied getVersion work

* masterserver - remove the C++ implementation (it can't be supported as state is now missing)

* Remove unnecessary SWIFT_CXX_REF_IMMORTAL annotations from Flow types

* Remove C++ implementation of CommitProxyVersionReplies - it's in Swift now

* [swift interop] remove more SWIFT_CXX_REF_IMMORTAL

* [swift interop] add SWIFT_CXX_IMMORTAL_SINGLETON_TYPE annotation for semanticly meaningful immortal uses

* rename SWIFT_CXX_REF_IMMORTAL -> UNSAFE_SWIFT_CXX_IMMORTAL_REF

* Move master server waitForPrev to swift

* =build fix linking swift in all modules

* =build single link option

* =cmake avoid manual math, just get "last" element from list

* implement Streams support (#18)

* [interop] update to new toolchain #6

* [interop] remove C++ vtable linking workarounds

* [interop] make MasterData proper reference counted SWIFT_CXX_REF_MASTERDATA

* [interop] use Swift array to pass UIDs to registerLastCommitProxyVersionReplies

* [interop] expose MasterServer actor to C++ without wrapper struct

* [interop] we no longer need expose on methods 🥳

* [interop] initial prototype of storing CheckedContinuation on the C++ side

* Example of invoking a synchronous swift function from a C++ unit test. (#21)

* move all "tests" we have in Swift, and priority support into real modules (#24)

* Make set continuation functions inline

* Split flow_swift into flow_swift and flow_swift_future to break circular dependency

* rename SwiftContinuationCallbackStruct to FlowCallbackForSwiftContinuation

* Future interop: use a method in a class template for continuation set call

* Revert "Merge pull request #22 from FoundationDB/cpp-continuation" (#30)

* Basic Swift Guide (#29)

Co-authored-by: Alex Lorenz <arphaman@gmail.com>

* Revert "Revert "Merge pull request #22 from FoundationDB/cpp-continuation" (#30)"

This reverts commit c025fe6258c4c4904d5e70cd549d408bb61f2802.

* Restore the C++ continuation, but it seems waitValue is broken for CInt somehow now

* disable broken tests - waitValue not accessible

* Streams can be async iterated over (#27)

Co-authored-by: Alex Lorenz <arphaman@gmail.com>

* remove work in progress things (#35)

* remove some not used (yet) code

* remove expose func for CInt, it's a primitive so we always have witness info (#37)

* +masterdata implement provideVersions in Swift (#36)

* serveLiveCommittedVersion in Swift (#38)

* Port updateLiveCommittedVersion to swift (#33)

Co-authored-by: Konrad `ktoso` Malawski <konrad_malawski@apple.com>

* Implement updateRecoveryData in Swift (#39)

Co-authored-by: Alex Lorenz <arphaman@gmail.com>

* Simplify flow_swift to avoid multiple targets and generate separate CheckedContinuation header

* Uncomment test which was blocked on extensions not being picked up (#31)

* [interop] Use a separate target for Swift-to-C++ header generation

* reduce boilerplate in future and stream support (#41)

* [interop] require interop v8 - that will fix linker issue (https://github.com/apple/swift/issues/62448)

* [interop] fix swift_stream_support.h Swift include

* [interop] bump up requirement to version 9

* [interop] Generalize the Flow.Optional -> Swift.Optional conversion using generics

* [WIP] masterServer func in Swift (#45)

* [interop] Try conforms_to with a SWIFT_CONFORMS_TO macro for Optional conformance (#49)

* [interop] include FlowOptionalProtocol source file when generating Flow_CheckedContinuation.h

This header generation step depends on the import of the C++ Flow module, which requires the presence of FlowOptionalProtocol

* conform Future to FlowFutureOps

* some notes

* move to value() so we can use discardable result for Flow.Void

* make calling into Swift async funcs nicer by returning Flow Futures

* [interop] hide initial use of FlowCheckedContinuation in flow.h to break dependency cycle

* [fdbserver] fix an EncryptionOpsUtils.h modularization issue (showed up with modularized libc++)

* Pass GCC toolchain using CMAKE_Swift_COMPILE_EXTERNAL_TOOLCHAIN to Swift's clang importer

* [interop] drop the no longer needed libstdc++ include directories

* [cmake] add a configuration check to ensure Swift can import C++ standard library

* [swift] include msgpack from msgpack_DIR

* [interop] make sure the FDB module maps have 'export' directive

* add import 'flow_swift' to swift_fdbserver_cxx_swift_value_conformance.swift

This is needed for CONFORMS_TO to work in imported modules

* make sure the Swift -> C++ manually bridged function signature matches generated signature

* [interop][workaround] force back use of @expose attribute before _Concurrency issue is fixed

* [interop] make getResolutionBalancer return a pointer to allow Swift to use it

We should revert back to a reference once compiler allows references again

* [interop] add a workaround for 'pop' being marked as unsafe in Swift

* masterserver.swift: MasterData returns the Swift actor pointer in an unsafe manner

* Add a 'getCopy' method to AsyncVar to make it more Swift friendly

* [interop] bump up the toolchain requirement

* Revert "[interop][workaround] force back use of @expose attribute before _Concurrency issue is fixed"

This reverts commit b01b271a76d1677bbb7c5c9c64cdad4b8b2b9612.

* [interop] add FIXME comments highlighting new issue workarounds

* [interop] adopt the new C++ interoperability compiler flag

* [interop] generate swift compile commands

* Do not deduplicate Swift compilation commands

* [interop] generate swift compile commands

* Do not deduplicate Swift compilation commands

* flow actorcompiler.h: add a SWIFT_ACTOR empty macro definition

This is needed to make the actor files parsable by clangd

* [cmake] add missing dependencies

* experimental cross compile

* [cmake] fix triple in cross-compiled cmake flags

* [interop] update to interop toolchain version 16

* [x-compile] add flags for cross-compiling boost

* cleanup x-compile cmake changes

* [cmake] fix typo in CMAKE_Swift_COMPILER_EXTERNAL_TOOLCHAIN config variable

* [interop] pass MasterDataActor from Swift to C++ and back to Swift

* [fdbserver] Swift->C++ header generation for FDBServer should use same module cache path

* Update swift_get_latest_toolchain.sh to fetch 5.9 toochains

* set HAVE_FLAG_SEARCH_PATHS_FIRST for cross compilation

* Resolve conflicts in net2/sim2/actors, can't build yet

* undo SWIFT_ACTOR changes, not necessary for merge

* guard c++ compiler flags with is_cxx_compile

* Update flow/actorcompiler/ActorParser.cs

Co-authored-by: Evan Wilde <etceterawilde@gmail.com>

* update the boost dependency

* Include boost directory from the container for Swift

* conform flow's Optional to FlowOptionalProtocol again

* Guard entire RocksDBLogForwarder.h with SSD_ROCKSDB_EXPERIMENTAL to avoid failing on missing rocksdb APIs

* remove extraneous merge marker

* [swift] update swift_test_streams.swifto to use vars in more places

* Add header guard to flow/include/flow/ThreadSafeQueue.h to fix moduralization issue

* Update net and sim impls

* [cmake] use prebuilt libc++ boost only when we're actually using libc++

* [fdbserver] Swift->C++ header generation for FDBServer should use same module cache path

* fixups after merge

* remove CustomStringConvertible conformance that would not be used

* remove self-caused deprecation warnings in future_support

* handle newly added task priority

* reformatting

* future: make value() not mutating

* remove FIXME, not needed anymore

* future: clarify why as functions

* Support TraceEvent in Swift

* Enable TraceEvent using a class wrapper in Swift

* prearing WITH_SWIFT flag

* wip disabled failing Go stuff

* cleanup WITH_SWIFT_FLAG and reenable Go

* wip disabled failing Go stuff

* move setting flag before printing it

* Add SWIFT_IDE_SETUP and cleanup guides and build a bit

* Revert "Wipe packet buffers that held serialized WipedString (#10018)"

This reverts commit e2df6e33029897360f8e11b3aea8fef97393a98c.

* [Swift] Compile workaround in KeyBackedRangeMap; default init is incorrect

* [interop] do not add FlowFutureOps conformance when building flow clang module for Flow checked continuation header pre-generation

* make sure to show  -DUSE_LIBCXX=OFF in readme

* readme updates

* do not print to stderr

* Update Swift and C++ code to build with latest Swift 5.9 toolchain now that we no longer support universal references and bridge the methods that take in a constant reference template parameter correctly

* Fix SERVER_KNOBS and enable use them for masterserver

* Bump to C++20, Swift is now able to handle it as well

* Put waitForPrev behind FLOW_WITH_SWIFT knob

* Forward declare updateLiveCommittedVersion

* Remove unused code

* fix wrong condition set for updateLiveCommittedVersion

* Revert "Revert "Wipe packet buffers that held serialized WipedString (#10018)""

This reverts commit 5ad8dce0525fb1844664ed2ccd7ba595db6913dd.

* Enable go-bindings in cmake

* Revert "Revert "Wipe packet buffers that held serialized WipedString (#10018)""

This reverts commit 5ad8dce0525fb1844664ed2ccd7ba595db6913dd.

* USE_SWIFT flag so we "build without swift" until ready to by default

* uncomment a few tests which were disabled during USE_SWIFT enablement

* the option is WITH_SWIFT, not USE

* formatting

* Fix masterserver compile error

* Fix some build errors.

How did it not merge cleanly? :/

* remove initializer list from constructor

* Expect Swift toolchain only if WITH_SWIFT is enabled

* Don't require Flow_CheckedContinuation when Swift is disabled

* Don't compile FlowCheckedContinuation when WITH_SWIFT=OFF

* No-op Swift macros

* More compile guards

* fix typo

* Run clang-format

* Guard swift/bridging include in fdbrpc

* Remove printf to pass the test

* Remove some more printf to avoid potential issues

TODO: Need to be TraceEvents instead

* Remove __has_feature(nullability) as its only used in Swift

* Don't use __FILENAME__

* Don't call generate_module_map outside WITH_SWIFT

* Add some more cmake stuff under WITH_SWIFT guard

* Some more guards

* Bring back TLSTest.cpp

* clang-format

* fix comment formatting

* Remove unused command line arg

* fix cmake formatting in some files

* Address some review comments

* fix clang-format error

---------

Co-authored-by: Alex Lorenz <arphaman@gmail.com>
Co-authored-by: Russell Sears <russell_sears@apple.com>
Co-authored-by: Evan Wilde <etceterawilde@gmail.com>
Co-authored-by: Alex Lorenz <aleksei_lorenz@apple.com>
Co-authored-by: Vishesh Yadav <vishesh_yadav@apple.com>
Co-authored-by: Vishesh Yadav <vishesh3y@gmail.com>
2023-06-02 16:09:28 -05:00

249 lines
7.5 KiB
Swift

/*
* clock_support.swift
*
* This source file is part of the FoundationDB open source project
*
* Copyright 2013-2022 Apple Inc. and the FoundationDB project authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import Flow
public struct FlowClock {
/// A flow point in time used for `FlowClock`.
public struct Instant: Codable, Sendable {
internal var _value: Swift.Duration
internal init(_value: Swift.Duration) {
self._value = _value
}
}
public init() {
}
}
extension Clock where Self == FlowClock {
/// A clock that measures time that always increments but does not stop
/// incrementing while the system is asleep.
///
/// try await Task.sleep(until: .now + .seconds(3), clock: .flow)
///
public static var flow: FlowClock {
return FlowClock()
}
}
extension FlowClock: Clock {
/// The current flow instant.
public var now: FlowClock.Instant {
FlowClock.now
}
/// The minimum non-zero resolution between any two calls to `now`.
public var minimumResolution: Swift.Duration {
let seconds = Int64(0)
var nanoseconds = Int64(0)
nanoseconds += Duration.milliseconds(100).nanoseconds
return .seconds(seconds) + .nanoseconds(nanoseconds)
}
/// The current flow instant.
public static var now: FlowClock.Instant {
var seconds = Int64(0)
var nanoseconds = Int64(0)
let nowDouble: Double = flow_gNetwork_now()
seconds += Duration.seconds(nowDouble).seconds
nanoseconds = Duration.seconds(nowDouble).nanoseconds
return FlowClock.Instant(_value: .seconds(seconds) + .nanoseconds(nanoseconds))
}
/// Suspend task execution until a given deadline within a tolerance.
/// If no tolerance is specified then the system may adjust the deadline
/// to coalesce CPU wake-ups to more efficiently process the wake-ups in
/// a more power efficient manner.
///
/// If the task is canceled before the time ends, this function throws
/// `CancellationError`.
///
/// This function doesn't block the underlying thread.
public func sleep(
until deadline: Instant,
tolerance: Swift.Duration? = nil
) async throws {
let (seconds, attoseconds) = deadline._value.components
_ = seconds // silence warning
let nanoseconds = attoseconds / 1_000_000_000
_ = nanoseconds // silence warning
let duration = FlowClock.now.duration(to: deadline)
let secondsDouble = Double(duration.seconds)
let nanosDouble = 0 // TODO: fix this handling of nanos from the deadline
_ = nanosDouble // silence warning
try await flow_gNetwork_delay(/*secondsDouble=*/secondsDouble, /*priority=*/TaskPriority.DefaultDelay).value()
}
public static func sleep(
for duration: Duration
) async throws {
try await Task.sleep(until: now + duration, clock: .flow)
}
}
extension FlowClock.Instant: InstantProtocol {
public static var now: FlowClock.Instant {
FlowClock.now
}
public func advanced(by duration: Swift.Duration) -> FlowClock.Instant {
return FlowClock.Instant(_value: _value + duration)
}
public func duration(to other: FlowClock.Instant) -> Swift.Duration {
other._value - _value
}
public func hash(into hasher: inout Hasher) {
hasher.combine(_value)
}
public static func ==(
_ lhs: FlowClock.Instant, _ rhs: FlowClock.Instant
) -> Bool {
return lhs._value == rhs._value
}
public static func <(
_ lhs: FlowClock.Instant, _ rhs: FlowClock.Instant
) -> Bool {
return lhs._value < rhs._value
}
@_alwaysEmitIntoClient
@inlinable
public static func +(
_ lhs: FlowClock.Instant, _ rhs: Swift.Duration
) -> FlowClock.Instant {
lhs.advanced(by: rhs)
}
@_alwaysEmitIntoClient
@inlinable
public static func +=(
_ lhs: inout FlowClock.Instant, _ rhs: Swift.Duration
) {
lhs = lhs.advanced(by: rhs)
}
@_alwaysEmitIntoClient
@inlinable
public static func -(
_ lhs: FlowClock.Instant, _ rhs: Swift.Duration
) -> FlowClock.Instant {
lhs.advanced(by: .zero - rhs)
}
@_alwaysEmitIntoClient
@inlinable
public static func -=(
_ lhs: inout FlowClock.Instant, _ rhs: Swift.Duration
) {
lhs = lhs.advanced(by: .zero - rhs)
}
@_alwaysEmitIntoClient
@inlinable
public static func -(
_ lhs: FlowClock.Instant, _ rhs: FlowClock.Instant
) -> Swift.Duration {
rhs.duration(to: lhs)
}
}
// ==== ----------------------------------------------------------------------------------------------------------------
// MARK: Duration conversion support
extension Swift.Duration {
public typealias Value = Int64
public var nanoseconds: Value {
let (seconds, attoseconds) = self.components
let sNanos = seconds * Value(1_000_000_000)
let asNanos = attoseconds / Value(1_000_000_000)
let (totalNanos, overflow) = sNanos.addingReportingOverflow(asNanos)
return overflow ? .max : totalNanos
}
/// The microseconds representation of the `TimeAmount`.
public var microseconds: Value {
self.nanoseconds / TimeUnit.microseconds.rawValue
}
/// The milliseconds representation of the `TimeAmount`.
public var milliseconds: Value {
self.nanoseconds / TimeUnit.milliseconds.rawValue
}
/// The seconds representation of the `TimeAmount`.
public var seconds: Value {
self.nanoseconds / TimeUnit.seconds.rawValue
}
public var isEffectivelyInfinite: Bool {
self.nanoseconds == .max
}
/// Represents number of nanoseconds within given time unit
public enum TimeUnit: Value {
case days = 86_400_000_000_000
case hours = 3_600_000_000_000
case minutes = 60_000_000_000
case seconds = 1_000_000_000
case milliseconds = 1_000_000
case microseconds = 1000
case nanoseconds = 1
public var abbreviated: String {
switch self {
case .nanoseconds: return "ns"
case .microseconds: return "μs"
case .milliseconds: return "ms"
case .seconds: return "s"
case .minutes: return "m"
case .hours: return "h"
case .days: return "d"
}
}
public func duration(_ duration: Int) -> Duration {
switch self {
case .nanoseconds: return .nanoseconds(Value(duration))
case .microseconds: return .microseconds(Value(duration))
case .milliseconds: return .milliseconds(Value(duration))
case .seconds: return .seconds(Value(duration))
case .minutes: return .seconds(Value(duration) * 60)
case .hours: return .seconds(Value(duration) * 60 * 60)
case .days: return .seconds(Value(duration) * 24 * 60 * 60)
}
}
}
}