apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-06-02 11:15:50 +08:00
Code Issues Packages Projects Releases Wiki Activity
14,449 Commits 35 Branches 318 Tags
a892eec57d
Commit Graph

14449 Commits

Author SHA1 Message Date
Vishesh Yadav
d7252da951 clang-format: Fix the TEST() macros which require comments in line 2021-03-10 16:50:53 -08:00
Vishesh Yadav
2cd3f45fd6 Merge remote-tracking branch 'apple/release-6.3' into master-format-final 
release-6.3 was recently merged, and there were two PRs which were
merged in between and got those changes in here. Hence, since all the
changes were in, discarded the incoming changes and accepted all
current.
 2021-03-10 16:50:44 -08:00
A.J. Beamon
b7614bad24 Update the multithreaded_client test to work with Python 3 2021-03-10 16:47:33 -08:00
A.J. Beamon
fe4ff420a3 Don't copy the external libraries if you aren't using more than one thread 2021-03-10 15:09:30 -08:00
Chaoguang Lin
513abfe04a Revert change on storageCacheServer 2021-03-10 14:09:10 -08:00
Markus Pilman
cfbaaa6001 Merge remote-tracking branch 'origin/release-6.2' into release-6.3 2021-03-10 14:30:47 -07:00
Markus Pilman
5b11c4c6d8
Merge pull request from sfc-gh-satherton/file-backup-async-rename 
File-based backup now uses async rename and larger writes
 2021-03-10 14:23:10 -07:00
Markus Pilman
2ba257e6f3
Merge pull request from sfc-gh-ajbeamon/merge-release-6.2-into-release-6.3 
Merge release 6.2 into release 6.3 (clang formatting change)
 2021-03-10 14:19:25 -07:00
A.J. Beamon
13e9755138 Merge branch 'release-6.2' into merge-release-6.2-into-release-6.3 
This discards all changes, as the only release-6.2 change that hasn't been merged is a formatting change.

# Conflicts:
#	FDBLibTLS/FDBLibTLSPolicy.cpp
#	bindings/c/ThreadCleanup.cpp
#	bindings/c/fdb_c.cpp
#	bindings/c/test/mako/mako.c
#	bindings/c/test/mako/mako.h
#	bindings/c/test/performance_test.c
#	bindings/c/test/ryw_benchmark.c
#	bindings/c/test/txn_size_test.c
#	bindings/flow/Tuple.cpp
#	bindings/flow/fdb_flow.actor.cpp
#	bindings/flow/fdb_flow.h
#	bindings/flow/tester/Tester.actor.cpp
#	bindings/flow/tester/Tester.actor.h
#	bindings/java/fdbJNI.cpp
#	fdbbackup/backup.actor.cpp
#	fdbcli/FlowLineNoise.actor.cpp
#	fdbcli/fdbcli.actor.cpp
#	fdbcli/linenoise/linenoise.c
#	fdbcli/linenoise/linenoise.h
#	fdbclient/AsyncFileBlobStore.actor.h
#	fdbclient/Atomic.h
#	fdbclient/BackupAgent.actor.h
#	fdbclient/BackupAgentBase.actor.cpp
#	fdbclient/BackupContainer.actor.cpp
#	fdbclient/BackupContainer.h
#	fdbclient/BlobStore.actor.cpp
#	fdbclient/BlobStore.h
#	fdbclient/ClientLogEvents.h
#	fdbclient/CommitTransaction.h
#	fdbclient/CoordinationInterface.h
#	fdbclient/DatabaseBackupAgent.actor.cpp
#	fdbclient/DatabaseConfiguration.cpp
#	fdbclient/DatabaseConfiguration.h
#	fdbclient/DatabaseContext.h
#	fdbclient/FDBTypes.h
#	fdbclient/FailureMonitorClient.actor.cpp
#	fdbclient/FailureMonitorClient.h
#	fdbclient/FileBackupAgent.actor.cpp
#	fdbclient/HTTP.actor.cpp
#	fdbclient/JSONDoc.h
#	fdbclient/KeyRangeMap.actor.cpp
#	fdbclient/KeyRangeMap.h
#	fdbclient/Knobs.cpp
#	fdbclient/ManagementAPI.actor.cpp
#	fdbclient/ManagementAPI.actor.h
#	fdbclient/MasterProxyInterface.h
#	fdbclient/MonitorLeader.actor.cpp
#	fdbclient/MonitorLeader.h
#	fdbclient/MultiVersionTransaction.actor.cpp
#	fdbclient/MultiVersionTransaction.h
#	fdbclient/MutationList.h
#	fdbclient/NativeAPI.actor.cpp
#	fdbclient/NativeAPI.actor.h
#	fdbclient/Notified.h
#	fdbclient/RYWIterator.cpp
#	fdbclient/ReadYourWrites.actor.cpp
#	fdbclient/ReadYourWrites.h
#	fdbclient/RunTransaction.actor.h
#	fdbclient/SnapshotCache.h
#	fdbclient/Status.h
#	fdbclient/StorageServerInterface.h
#	fdbclient/SystemData.cpp
#	fdbclient/SystemData.h
#	fdbclient/TaskBucket.actor.cpp
#	fdbclient/ThreadSafeTransaction.actor.cpp
#	fdbclient/VersionedMap.h
#	fdbclient/WriteMap.h
#	fdbclient/rapidjson/document.h
#	fdbmonitor/fdbmonitor.cpp
#	fdbrpc/ActorFuzz.actor.cpp
#	fdbrpc/ActorFuzz.h
#	fdbrpc/AsyncFileCached.actor.cpp
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/AsyncFileKAIO.actor.h
#	fdbrpc/AsyncFileWriteChecker.h
#	fdbrpc/FailureMonitor.actor.cpp
#	fdbrpc/FailureMonitor.h
#	fdbrpc/FlowTests.actor.cpp
#	fdbrpc/FlowTransport.actor.cpp
#	fdbrpc/FlowTransport.h
#	fdbrpc/LoadBalance.actor.h
#	fdbrpc/Locality.h
#	fdbrpc/MultiInterface.h
#	fdbrpc/Net2FileSystem.cpp
#	fdbrpc/Platform.cpp
#	fdbrpc/QueueModel.cpp
#	fdbrpc/QueueModel.h
#	fdbrpc/RangeMap.h
#	fdbrpc/Replication.h
#	fdbrpc/ReplicationUtils.cpp
#	fdbrpc/ReplicationUtils.h
#	fdbrpc/Stats.actor.cpp
#	fdbrpc/Stats.h
#	fdbrpc/crc32c.cpp
#	fdbrpc/dsltest.actor.cpp
#	fdbrpc/fdbrpc.h
#	fdbrpc/generated-constants.cpp
#	fdbrpc/libcoroutine/Coro.c
#	fdbrpc/libeio/eio.c
#	fdbrpc/libeio/xthread.h
#	fdbrpc/sim2.actor.cpp
#	fdbrpc/simulator.h
#	fdbrpc/zlib/gzread.c
#	fdbserver/ApplyMetadataMutation.cpp
#	fdbserver/ApplyMetadataMutation.h
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/ClusterRecruitmentInterface.h
#	fdbserver/ConflictSet.h
#	fdbserver/CoordinatedState.actor.cpp
#	fdbserver/Coordination.actor.cpp
#	fdbserver/CoordinationInterface.h
#	fdbserver/CoroFlow.actor.cpp
#	fdbserver/DBCoreState.h
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/DataDistribution.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/DataDistributionTracker.actor.cpp
#	fdbserver/DataDistributorInterface.h
#	fdbserver/DeltaTree.h
#	fdbserver/DiskQueue.actor.cpp
#	fdbserver/FDBExecHelper.actor.cpp
#	fdbserver/FDBExecHelper.actor.h
#	fdbserver/IDiskQueue.h
#	fdbserver/IKeyValueStore.h
#	fdbserver/IPager.h
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/KeyValueStoreSQLite.actor.cpp
#	fdbserver/Knobs.h
#	fdbserver/LeaderElection.actor.cpp
#	fdbserver/LeaderElection.h
#	fdbserver/LogRouter.actor.cpp
#	fdbserver/LogSystem.h
#	fdbserver/LogSystemConfig.h
#	fdbserver/LogSystemDiskQueueAdapter.actor.cpp
#	fdbserver/LogSystemDiskQueueAdapter.h
#	fdbserver/LogSystemPeekCursor.actor.cpp
#	fdbserver/MasterInterface.h
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/MoveKeys.actor.cpp
#	fdbserver/OldTLogServer_4_6.actor.cpp
#	fdbserver/OldTLogServer_6_0.actor.cpp
#	fdbserver/QuietDatabase.actor.cpp
#	fdbserver/Ratekeeper.actor.cpp
#	fdbserver/RatekeeperInterface.h
#	fdbserver/Resolver.actor.cpp
#	fdbserver/ResolverInterface.h
#	fdbserver/Restore.actor.cpp
#	fdbserver/RestoreInterface.h
#	fdbserver/ServerDBInfo.h
#	fdbserver/SimulatedCluster.actor.cpp
#	fdbserver/SkipList.cpp
#	fdbserver/Status.actor.cpp
#	fdbserver/Status.h
#	fdbserver/StorageMetrics.actor.h
#	fdbserver/TLogInterface.h
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/TagPartitionedLogSystem.actor.cpp
#	fdbserver/VersionedBTree.actor.cpp
#	fdbserver/WaitFailure.actor.cpp
#	fdbserver/WaitFailure.h
#	fdbserver/WorkerInterface.actor.h
#	fdbserver/fdbserver.actor.cpp
#	fdbserver/masterserver.actor.cpp
#	fdbserver/networktest.actor.cpp
#	fdbserver/sqlite/btree.c
#	fdbserver/storageserver.actor.cpp
#	fdbserver/tester.actor.cpp
#	fdbserver/worker.actor.cpp
#	fdbserver/workloads/ApiCorrectness.actor.cpp
#	fdbserver/workloads/AsyncFileCorrectness.actor.cpp
#	fdbserver/workloads/AtomicOps.actor.cpp
#	fdbserver/workloads/AtomicRestore.actor.cpp
#	fdbserver/workloads/AtomicSwitchover.actor.cpp
#	fdbserver/workloads/BackupCorrectness.actor.cpp
#	fdbserver/workloads/BackupToDBCorrectness.actor.cpp
#	fdbserver/workloads/BackupToDBUpgrade.actor.cpp
#	fdbserver/workloads/BulkSetup.actor.h
#	fdbserver/workloads/ClientTransactionProfileCorrectness.actor.cpp
#	fdbserver/workloads/ConfigureDatabase.actor.cpp
#	fdbserver/workloads/ConsistencyCheck.actor.cpp
#	fdbserver/workloads/Cycle.actor.cpp
#	fdbserver/workloads/FuzzApiCorrectness.actor.cpp
#	fdbserver/workloads/KVStoreTest.actor.cpp
#	fdbserver/workloads/KillRegion.actor.cpp
#	fdbserver/workloads/LowLatency.actor.cpp
#	fdbserver/workloads/MachineAttrition.actor.cpp
#	fdbserver/workloads/Mako.actor.cpp
#	fdbserver/workloads/Performance.actor.cpp
#	fdbserver/workloads/ReadWrite.actor.cpp
#	fdbserver/workloads/RemoveServersSafely.actor.cpp
#	fdbserver/workloads/Rollback.actor.cpp
#	fdbserver/workloads/Sideband.actor.cpp
#	fdbserver/workloads/SlowTaskWorkload.actor.cpp
#	fdbserver/workloads/StatusWorkload.actor.cpp
#	fdbserver/workloads/VersionStamp.actor.cpp
#	fdbserver/workloads/Watches.actor.cpp
#	fdbserver/workloads/WriteDuringRead.actor.cpp
#	fdbservice/FDBService.cpp
#	fdbservice/ServiceBase.cpp
#	flow/Arena.h
#	flow/Deque.cpp
#	flow/Deque.h
#	flow/Error.cpp
#	flow/Error.h
#	flow/FastAlloc.cpp
#	flow/FastAlloc.h
#	flow/FileTraceLogWriter.cpp
#	flow/FileTraceLogWriter.h
#	flow/IRandom.h
#	flow/IThreadPool.cpp
#	flow/IThreadPool.h
#	flow/IndexedSet.actor.h
#	flow/IndexedSet.cpp
#	flow/IndexedSet.h
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/Net2.actor.cpp
#	flow/Net2Packet.cpp
#	flow/ObjectSerializer.h
#	flow/Platform.cpp
#	flow/Profiler.actor.cpp
#	flow/ProtocolVersion.h
#	flow/SignalSafeUnwind.cpp
#	flow/SignalSafeUnwind.h
#	flow/SystemMonitor.cpp
#	flow/TDMetric.actor.h
#	flow/TLSConfig.actor.cpp
#	flow/ThreadHelper.actor.h
#	flow/ThreadPrimitives.h
#	flow/Trace.cpp
#	flow/Trace.h
#	flow/Util.h
#	flow/error_definitions.h
#	flow/flow.cpp
#	flow/flow.h
#	flow/genericactors.actor.cpp
#	flow/genericactors.actor.h
#	flow/network.cpp
#	flow/network.h
#	flow/serialize.h
 2021-03-10 11:05:24 -08:00
Vishesh Yadav
af36e52fdf Merge branch 'release-6.3-pre-format' into master-format 
This merges release-6.3 branch right before it was fully formatted.
There were quite a few conflicts that are resolved here. CoroFlow had
a check for OOM errors introduced in 6.3, but didn't seem applicable in
the new implmentation which seems to use boost.
 2021-03-10 10:40:53 -08:00
FDB Formatster
df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Evan Tschannen
70a35c4888 Merge branch 'release-6.2' into feature-fix-longest-time 
# Conflicts:
#	fdbserver/storageserver.actor.cpp
 2021-03-10 10:15:55 -08:00
Vishesh Yadav
2bb4f2e59f Merge branch 'release-6.3-pre-format' into master-format 
This merges release-6.3 branch right before it was fully formatted.
There were quite a few conflicts that are resolved here. CoroFlow had
a check for OOM errors introduced in 6.3, but didn't seem applicable in
the new implmentation which seems to use boost.
 2021-03-10 09:37:41 -08:00
Vishesh Yadav
3fbe7e69fa Add TraceEvent to warn tLogs which has not joined after timeout 
During recovery, wait for TLOG_SLOW_REJOIN_WARN_TIMEOUT_SECS and
log the tLogs that has not joined yet.
 2021-03-09 18:57:46 -08:00
Markus Pilman
a80c290cad
Merge pull request from sfc-gh-anoyes/anoyes/boost-valgrind 
Inform boost context of valgrind usage
 2021-03-09 15:17:13 -07:00
Andrew Noyes
ccc8404faa Disable go bindings for sanitizer builds 2021-03-09 20:55:41 +00:00
Chaoguang Lin
f99b6b04fe Update comments for cache consistency check, refactor some part of the code 2021-03-09 12:23:38 -08:00
Chaoguang Lin
7439ec227b Add the test that sets invalid values 2021-03-09 12:21:18 -08:00
Chaoguang Lin
8b578185f4 Enable traces 2021-03-09 12:18:11 -08:00
Markus Pilman
58550f9e37
Merge pull request from sfc-gh-clin/fix-trace-name-inconsistency 
Fix base trace event attribute name inconsistency
 2021-03-09 13:12:00 -07:00
Evan Tschannen
060885d9ad removed changes to downloads.rst applied clang format 2021-03-09 11:46:54 -08:00
Scott Fines
85a6e65e16
Merge pull request from scottfines/instrumented_java 
Add the ability to instrument java driver actions
 2021-03-09 13:46:31 -06:00
Andrew Noyes
fb8d021995 Pass SANITIZER_LINK_OPTIONS to CGO 2021-03-09 17:54:34 +00:00
Scott Fines
5f1fd54567
Merge pull request from scottfines/buffer_defaults 
Better iteration_progression size defaults
 2021-03-09 11:47:25 -06:00
A.J. Beamon
7535261184
Fix typo 2021-03-09 09:43:42 -08:00
Scott Fines
bdc888ba0a fixing formatting errors 2021-03-09 11:34:15 -06:00
Nim Wijetunga
013867bd66
Merge pull request from apple/master 
Merge Master
 2021-03-09 12:33:26 -05:00
Andrew Noyes
9bdd8789e9
Merge pull request from sfc-gh-mpilman/bugfix/alternatives-deb 
Fix Debian problems with versioned packages
 2021-03-09 08:53:03 -08:00
Scott Fines
2db95bebf9
Merge branch 'release-6.3' into buffer_defaults 2021-03-09 08:53:12 -06:00
Scott Fines
329d6052b6 making times record counts too 2021-03-09 08:41:10 -06:00
Steve Atherton
7066513109
Merge pull request from sfc-gh-anoyes/anoyes/clang-tidy-warning 
Add ASSERT to help clang-tidy
 2021-03-08 18:55:26 -08:00
Chaoguang Lin
35e6e50c21 Merge branch 'release-6.3' of github.com:apple/foundationdb into fix-trace-name-inconsistency 2021-03-08 18:12:49 -08:00
Andrew Noyes
8766bb44b1 Address review comment 2021-03-09 01:40:59 +00:00
Andrew Noyes
b5084c634d Inform boost context of valgrind usage 
This fixes the "client switching stacks?" warning that valgrind is
currently issuing.

Also fix a memory error that currently doesn't manifest because of the
way FastAllocator works (if you free a small buffer and then immediately
allocate the same size buffer in FastAllocator you always get the same
buffer back). You can see it if you set FDB_VALGRIND_PRECISE= (causes
FastAllocator to use malloc)

Also minor refactoring and cleanup of CoroFlow.actor.cpp

Also (probably) fix the other sanitizer builds, although I didn't test
that
 2021-03-09 01:33:32 +00:00
Russell Sears
2b6da2ef28
Merge pull request from ammolitor/clang_format_the_world/6.3 
apply clang-format to *.c, *.cpp, *.h, *.hpp files
 2021-03-08 15:52:30 -08:00
Chaoguang Lin
9645f489e6 Fix base trace event name inconsistency 2021-03-08 15:20:50 -08:00
Russell Sears
36bf68ed2e
Merge branch 'release-6.3' into clang_format_the_world/6.3 2021-03-08 14:40:52 -08:00
Andrew Noyes
5ecb1dfcba Move ASSERT to right before dereference 2021-03-08 22:37:30 +00:00
Nim Wijetunga
f74f9aa770
Merge pull request from apple/master 
Merge Master
 2021-03-08 15:46:40 -05:00
Steve Atherton
6df9393668
Merge pull request from sfc-gh-anoyes/anoyes/server-handshake-threads-default 
Change default for TLS_SERVER_HANDSHAKE_THREADS to 0
 2021-03-08 12:28:49 -08:00
Scott Fines
df6faa089a Addressing PR comments 2021-03-08 14:10:20 -06:00
Scott Fines
a9324ebbf6 Adjust default native buffer sizes to be more efficient 2021-03-08 13:31:44 -06:00
Andrew Noyes
7188c7ddfb Add ASSERT to help clang-tidy 
If n == nullptr here, then we'll dereference a nullptr. Add an assert.
 2021-03-08 18:50:28 +00:00
Andrew Noyes
a7ccb6669d Change Net2::connect to only call initTLS for tls 2021-03-08 16:27:04 +00:00
Scott Fines
6e32eb75dc Adding a reference implementation of EventKeeper 2021-03-08 09:58:31 -06:00
Scott Fines
e080e989ac Better approach to non-locked stats gathering (including knowing what thread will operate) 2021-03-08 09:52:02 -06:00
Scott Fines
6c4ce1769d Moving external calls outside of the pointer lock 2021-03-08 08:46:06 -06:00
Steve Atherton
e9c9bd5299 Merge commit 'e2d3ddee2cb4da0bba65346d29ad2212fbce16f0' into file-backup-async-rename 
# Conflicts:
#	fdbclient/BackupContainer.actor.cpp
#	fdbclient/FileBackupAgent.actor.cpp
#	fdbclient/Knobs.cpp
#	fdbrpc/AsyncFileNonDurable.actor.h
#	fdbrpc/sim2.actor.cpp
 2021-03-08 02:00:40 -08:00
Steve Atherton
318b862aa3 Apply clang-format to backup changes. 2021-03-08 01:49:29 -08:00
Steve Atherton
a397d8625f AsyncFileNonDurable now has a flag for whether or not to emulate AIO mode, so the previous behavior limits are preserved when this flag is set. AsyncFileDurable writes which are selected to be durable are again done in a single write, and the logic to handle non-aligned writes for non-AIO mode is more clear and commented. In simulation, backup files on local filesystems are written with random buffer sizes. Backup container unit test now uses more random file sizes and limits memory consumption for all files being written in parallel. 2021-03-07 22:21:10 -08:00