apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-05-16 10:52:20 +08:00
Code Issues Packages Projects Releases Wiki Activity
21,307 Commits 35 Branches 318 Tags
Commit Graph

190 Commits

Author SHA1 Message Date
Lukas Joswiak
88557d9169 Simplify function call when transaction is null 2022-06-22 14:50:17 -07:00
Andrew Noyes
0fea3fb731
Save a bunch of copies in the trace thread (#7392) 
Currently, a std::string is copied unnecessarily for every key and value
in a trace event.

This actually showed up in a jemalloc heap profile while I was
investigating something unrelated. I was surprised to see it since these
allocations should have a very short lifetime.
 2022-06-15 12:29:15 -07:00
Vaidas Gasiunas
4d6ec7663f Close file at the end of runNetwork rather than in stopNetwork in order to avoid a race condition, 
also in order to continue tracing the pending network thread activity.
Poll event throttler only in the network thread in order to avoid a race condition.
 2022-05-31 13:59:09 +02:00
Markus Pilman
85757eb47c
Update flow/Trace.cpp 2022-04-20 15:30:42 -06:00
Markus Pilman
f7a8ebf818
Update flow/Trace.cpp 
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
 2022-04-20 11:55:01 -06:00
Markus Pilman
d4ee7be1d7 Reduce excessive tracing and fail after 1M traces 2022-04-19 21:11:51 -06:00
sfc-gh-tclinkenbeard
a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
A.J. Beamon
d45bae11b7 Fix substring call when reporting details about a trace event that was longer than the legal limit. 2022-03-04 13:59:49 -08:00
A.J. Beamon
f6b24cd644 Add some extra print debugging if we overflow the trace buffer. 2022-03-01 09:04:15 -08:00
A.J. Beamon
250a88e682 Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement. 2022-02-24 12:25:52 -08:00
Xiaoxi Wang
6dc5921575
createdTime based storage wiggler (#6219) 
* add storagemetadata

* add StorageWiggler;

* fix serverMetadataKey bug

* add metadata tracker in storage tracker

* finish StorageWiggler

* update next storage ID

* change pid to server id

* write metadata when seed SS

* add status json fields

* remove pid based ppw iteration

* fix time expression

* fix tss metadata nonexistence; fix transaction retry when retrieving metadata

* fix checkMetadata bug when store type is wrong

* fix remove storage status json

* format code

* refactor updateNextWigglingStoragePID

* seperate storage metadata tracker and store type tracker

* rename pid

* wiggler stats

* fix completion between waitServerListChange and storageRecruiter

* solve review comments

* rename system key

* fix database lock timeout by adding lock_aware

* format code

* status json

* resolve code format/naming comments

* delete expireNow; change PerpetualStorageWiggleID's value to KeyBackedObjectMap<UID, StorageWiggleValue>

* fix omit start rount

* format code

* status json reset

* solve status json format

* improve status json latency; replace binarywriter/reader to objectwriter/reader; refactor storagewigglerstats transactions

* status timestamp
 2022-02-04 15:04:30 -08:00
A.J. Beamon
b9581fb768 Convert variable to const& 2022-01-04 09:16:26 -08:00
A.J. Beamon
af931ad989 Add support for universal trace fields that apply to all events in a process. Add a universal field on clients called ClientDescription that identifies the client logging the event. 2021-12-10 15:03:14 -08:00
A.J. Beamon
b8bd89f88d Shorten the name of external client threads. Add a thread name for trace logging threads. 2021-11-29 09:57:10 -08:00
A.J. Beamon
77c3122391 Slight modification to comment to remove the indication that we flush the trace log explicitly. 2021-09-27 13:17:00 -07:00
A.J. Beamon
0fc3cf44dc When we have a large trace log buffer in simulation, that suggests we may be stuck in a loop that prevents us from running the trace flush. Detect when this has happened and fail the test early, which allows us to do the flush and have logging for the offending loop. 2021-09-27 13:17:00 -07:00
FDB Formatster
2c788c233d apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-08-27 17:07:47 -07:00
Andrew Noyes
7706b47085 Fix another TSAN warning 
```
  Read of size 1 at 0x7f02082ac578 by main thread:
    #0 TraceLog::close() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/Trace.cpp:487:7 (libfdb_c.so+0xf4928c)
    #1 closeTraceFile() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/Trace.cpp:759:13 (libfdb_c.so+0xf3992a)
    #2 stopNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/NativeAPI.actor.cpp:2052:2 (libfdb_c.so+0x58315c)
    #3 ThreadSafeApi::stopNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:466:2 (libfdb_c.so+0xce8ad3)
    #4 MultiVersionApi::stopNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/MultiVersionTransaction.actor.cpp:1756:20 (libfdb_c.so+0x4f7da3)
    #5 fdb_stop_network /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/fdb_c.cpp:134:2 (libfdb_c.so+0x4c84e5)
    #6 main /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/test/unit/trace_partial_file_suffix_test.cpp:93:12 (trace_partial_file_suffix_test+0x40b3d0)

  Previous write of size 1 at 0x7f02082ac578 by thread T1 (mutexes: write M100):
    #0 TraceLog::open(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, unsigned long, unsigned long, Optional<NetworkAddress>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/Trace.cpp:344:10 (libfdb_c.so+0xf48aaa)
    #1 openTraceFile(NetworkAddress const&, unsigned long, unsigned long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/Trace.cpp:741:13 (libfdb_c.so+0xf3828c)
    #2 Database::createDatabase(Reference<ClusterConnectionFile>, int, IsInternal, LocalityData const&, DatabaseContext*) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/NativeAPI.actor.cpp:1695:4 (libfdb_c.so+0x57d57e)
    #3 ThreadSafeDatabase::ThreadSafeDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int)::$_7::operator()() const /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:124:8 (libfdb_c.so+0xcf018c)
    #4 internal_thread_helper::DoOnMainThreadVoidActorState<ThreadSafeDatabase::ThreadSafeDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int)::$_7, internal_thread_helper::DoOnMainThreadVoidActor<ThreadSafeDatabase::ThreadSafeDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int)::$_7> >::a_body1cont1(Void const&, int) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/flow/ThreadHelper.actor.h:45:4 (libfdb_c.so+0xcf018c)
    #5 internal_thread_helper::DoOnMainThreadVoidActorState<ThreadSafeDatabase::ThreadSafeDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int)::$_7, internal_thread_helper::DoOnMainThreadVoidActor<ThreadSafeDatabase::ThreadSafeDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int)::$_7> >::a_body1when1(Void const&, int) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/flow/ThreadHelper.actor.g.h:148:15 (libfdb_c.so+0xcf018c)
    #6 internal_thread_helper::DoOnMainThreadVoidActorState<ThreadSafeDatabase::ThreadSafeDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int)::$_7, internal_thread_helper::DoOnMainThreadVoidActor<ThreadSafeDatabase::ThreadSafeDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int)::$_7> >::a_callback_fire(ActorCallback<internal_thread_helper::DoOnMainThreadVoidActor<ThreadSafeDatabase::ThreadSafeDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int)::$_7>, 0, Void>*, Void const&) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/flow/ThreadHelper.actor.g.h:169:4 (libfdb_c.so+0xcefdef)
    #7 ActorCallback<internal_thread_helper::DoOnMainThreadVoidActor<ThreadSafeDatabase::ThreadSafeDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int)::$_7>, 0, Void>::fire(Void const&) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/flow.h:1074:78 (libfdb_c.so+0xcefdef)
    #8 void SAV<Void>::send<Void>(Void&&) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/flow.h:462:23 (libfdb_c.so+0x4dd542)
    #9 void Promise<Void>::send<Void>(Void&&) const /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/flow.h:699:8
    ```
 2021-08-09 14:06:38 -07:00
Andrew Noyes
39eff8c569 Add trace_partial_file_suffix network option 2021-08-03 09:43:42 -07:00
sfc-gh-tclinkenbeard
c74047c665 Merge remote-tracking branch 'origin/master' into fix-more-clang-warnings 2021-07-28 11:51:02 -07:00
Steve Atherton
507c1f11e3 Add .log() to bare TraceEvent() invocations without any .detail()s to avoid clang-tidy warning about immediate destruction of object without use. 2021-07-26 19:55:10 -07:00
sfc-gh-tclinkenbeard
b20e02ca25 Fix more -Wreorder-ctor warnings across several files 2021-07-24 11:43:19 -07:00
sfc-gh-tclinkenbeard
3442ebd3b7 Fix more -Wreorder-ctor warnings across many files 2021-07-24 11:20:51 -07:00
Russell Sears
18cde28d6e Add a ThreadID field to each TraceEvent. This is needed for multithreaded client debugging, and should also be helpful for multithreaded storage engines, SSL handshake threadpools, and so on 2021-06-30 14:31:53 -07:00
sfc-gh-tclinkenbeard
399c2c96f0 Remove unnecessary std::string copies from flow 2021-06-09 11:40:01 -07:00
sfc-gh-tclinkenbeard
2bcfbd6816 Move TraceEvent::setErrorKind implementation to cpp file 2021-05-03 11:10:43 -07:00
sfc-gh-tclinkenbeard
13a6c4cf06 Add ErrorKind field to SevError trace events 2021-05-03 11:10:38 -07:00
FDB Formatster
df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Markus Pilman
2d84e5e1f1
Merge pull request #4254 from sfc-gh-tclinkenbeard/capture-forked-process-output 
Capture forked process output in traces
 2021-02-24 14:34:24 -07:00
Steve Atherton
76a4293d1d Merge branch 'release-6.3' 
# Conflicts:
#	fdbserver/fdbserver.actor.cpp
 2021-02-15 02:13:06 -08:00
Steve Atherton
f4c9b88908 Merge branch 'release-6.2' into release-6.3 
# Conflicts:
#	cmake/CompileBoost.cmake
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	flow/Error.cpp
 2021-02-15 02:05:03 -08:00
Andrew Noyes
4e184fe236 Fix memory errors 2021-02-11 02:58:21 +00:00
sfc-gh-tclinkenbeard
acac02587d Trace output from forked processes 2021-01-29 01:31:26 -08:00
sfc-gh-tclinkenbeard
5059228e9e Use unique_ptr for TraceEvent::tmpEventMetric 2020-12-08 09:09:32 -08:00
Andrew Noyes
7fbc4d7391 Resolve conflicts 2020-12-04 23:58:42 +00:00
Andrew Noyes
877997632d Merge branch 'release-6.3' into anoyes/merge-release-6.3-master 
Include conflict markers for review purposes
 2020-12-04 01:38:07 +00:00
Andrew Noyes
dc2bac5670 Resolve conflicts 2020-11-24 19:09:42 +00:00
Andrew Noyes
1f541f02be Merge branch 'anoyes/merge-6.2-to-6.3' into anoyes/release-6.3-merge 
Merge, leaving conflict markers for now
 2020-11-24 16:55:34 +00:00
David Youngworth
b1256b5dcd Address review comments, simplify DateTime 2020-11-18 16:55:47 -08:00
David Youngworth
5ade54f767 Fix windows build, DateTime to use UTC 2020-11-18 16:55:12 -08:00
David Youngworth
490fe61032 Fix bug in rolled Trace code 2020-11-18 16:55:06 -08:00
David Youngworth
50e515c29a Add DateTime to trace, initial commit 2020-11-18 16:54:49 -08:00
David Youngworth
d64cf8b9e3 Merge branch 6.3 into master 2020-11-17 11:22:45 -08:00
David Youngworth
d0391db862 Merge branch 'release-6.2' into release-6.3 2020-11-16 10:15:23 -08:00
sfc-gh-tclinkenbeard
392f18a2de Fix retrieveTraceLogIssues function name 2020-11-04 22:39:56 -08:00
Russell Sears
32c87bbb33 Lightweight, power of two spaced histogram implementation + automatic reporting 2020-11-02 11:13:16 -08:00
Xiaoge Su
ac4bebbfad Remove magic number 5 in severity eventsCount 2020-11-01 22:13:55 -08:00
A.J. Beamon
0a1478949a
Merge pull request #3892 from TheBenCollins/log-batch-priority-filter 
Add a check against the current minimum priority for TraceBatch logging
 2020-10-19 13:02:35 -07:00
Ben Collins
6166f7c99c Add a check against the current minimum priority for TraceBatch logging 2020-10-12 09:57:57 -04:00
sfc-gh-tclinkenbeard
a9607bdcec Explicitly seal classes that inherit but aren't inherited from 2020-10-07 21:58:24 -07:00