35 Commits

Author SHA1 Message Date
Andrew Noyes
96cbfe668c
Fix flaky ctest tests (#6310)
* Use localhost cluster for trace_partial_file_suffix_test

This way we get a predictable 127.0.0.1 in the trace file name

* Skip suspend test of pidof is not available

* Avoid writing to closed trace log

calling fdb_network_stop sends a "close" message to the trace thread,
but the network thread might can still be running and sending "flush"
messages to the network thread. This change basically ignores any
flushes that come after a close.

* Ensure unique ports for multi-process tests
2022-01-28 13:16:44 -08:00
Andrew Noyes
a77fcb5448 Finalize partial trace files during trace clean up 2021-08-03 13:26:48 -07:00
Andrew Noyes
39eff8c569 Add trace_partial_file_suffix network option 2021-08-03 09:43:42 -07:00
sfc-gh-tclinkenbeard
b9a22a61ef Fix many -Wreorder-ctor warnings 2021-07-23 17:33:18 -07:00
sfc-gh-tclinkenbeard
399c2c96f0 Remove unnecessary std::string copies from flow 2021-06-09 11:40:01 -07:00
FDB Formatster
df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
A.J. Beamon
aaf0a9aa7b Merge branch 'release-6.3' into merge-release-6.3-into-master
# Conflicts:
#	build/docker-compose.yaml
#	cmake/ConfigureCompiler.cmake
#	fdbclient/FileBackupAgent.actor.cpp
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/IAsyncFile.h
#	fdbrpc/IRateControl.h
#	fdbrpc/simulator.h
#	fdbserver/KeyValueStoreSQLite.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbservice/ServiceBase.cpp
2021-02-08 12:58:34 -08:00
Christophe Chevalier
ff0667b28d winbuild: define BOOST_USE_WINDOWS_H, WIN32_LEAN_AND_MEAN and NOMINMAX to fix various macro expansions issues
- NOMINMAX to fix confusion between std::max(..) and max(..) macro redefinition
- BOOST_USE_WINDOWS_H to prevent boost from redefining win32 APIs
- WIN32_LEAN_AND_MEAN to fix include ordering issues with winsock.h
2021-01-30 15:15:02 +01:00
sfc-gh-tclinkenbeard
b8a55dd097 Use unique_ptr for PImpl 2020-12-08 09:09:32 -08:00
sfc-gh-tclinkenbeard
0ac08f6a9b Replace NULL with nullptr in flow 2020-09-20 11:31:49 -07:00
Markus Pilman
0fbe7101c3 Revert "Revert "Request tracing""
This reverts commit 327cc31e354d80ca00003a2673bd19a6760285d4.
2020-07-07 10:06:13 -06:00
Jingyu Zhou
327cc31e35
Revert "Request tracing" 2020-06-16 12:32:42 -07:00
Markus Pilman
f6c50c60f3 Fix Windows build failure 2020-06-10 15:31:37 -07:00
Markus Pilman
1453ca6303 Added log writer 2020-05-05 17:58:41 -07:00
Xin Dong
5967ef5eab Added back the changes that report trace log flush failures and fix the random crash 2020-03-12 14:34:19 -07:00
Xin Dong
39610d15f8 Revert this change since it somehow introduced a random crash detected on circus 2020-03-04 16:14:38 -08:00
Xin Dong
f20619c9fb Resolve review comments. Changed how issues got cleared 2020-02-25 15:39:51 -08:00
Xin Dong
090c89e90a Addressed review comments. Fix the bug where issues on a worker may be wrongly cleared by subsequent GetDBinfo request. 2020-02-25 15:39:38 -08:00
Xin Dong
1c346fcfb0 Added the new issues into Status Schema. Remove the issue reporting in lastError since:
- If the issue string contains the error number, status schema needs to be super verbose to include all possible issue strings
- If the issue string does not contain the error number, the generic issue string can be pretty useless.

Thus now specific issues are being reported before calling lastError
2020-02-25 15:38:14 -08:00
Xin Dong
39c92c9cce Update flow/FileTraceLogWriter.cpp
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2020-02-25 15:38:14 -08:00
Xin Dong
f4f860bfa8 Changed issue reporting to be thread safe. Also changed the liveness ping to be thread safe. 2020-02-25 15:38:14 -08:00
Xin Dong
a6580dc15f Added the ability to ping a trace log writer thread and the monitoring in worker.actor.cpp. The current solution is simple a loose check. We can change this to be accurate check by using 'pthread_kill(writer_thread, 0)' 2020-02-25 15:37:53 -08:00
Xin Dong
0b0414fb94 Addressded review comments. Change the issue reporting from 'ITraceLogWriter' to be a more generic way. 2020-02-25 15:37:53 -08:00
Xin Dong
034dfe5e42 Now the inability to flush trace logs will be reported to both 'stderr' and also the status json object.
- Since the first flush failure, if the accumulated consecutive failure count exceeds the value defined in knobs, it will trigger the current worker process to report this issue via the 'GetServerDBInfo' interface of the cluster controler
    - A successful flush will reset the accumulated counter.
    Notice that the current solution does not take the time into consideration. The assumption is that flush failures tend to only happen in a clustered manner. The intermittent, but short, periods of flush failures are not considered as a problem since the memory pressure built by them should be negligible.
2020-02-25 15:37:32 -08:00
Andrew Noyes
5cef65b6c4
Update flow/FileTraceLogWriter.cpp
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-07-22 16:38:18 -07:00
mpilman
de4fe4d640 Remove dead code and revert back to base 10
there can be at most 2^32 files with a distinct index. Therefore
`log10(index) < 10` - this means base 10 is good enough.
2019-07-22 16:20:03 -07:00
Markus Pilman
a48bdc0095
Update flow/FileTraceLogWriter.cpp
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-07-22 14:16:34 -07:00
mpilman
c26c68d6d2 Address review comments 2019-07-22 11:45:05 -07:00
mpilman
a66dc937fc Make trace files lexicographically ordered
This resolves #1825

The basic idea is that we prefix every sequence number with its with,
separated by a dot. That way the filename length isn't increased
drastically (unlike using a fixed whidth number with 0-padding)
and the file names will be lexicographically ordered.
2019-07-22 10:10:47 -07:00
Alex Miller
1f02cd30e2 Mark all opened files as close-on-exec. 2019-05-13 16:05:49 -10:00
A.J. Beamon
85b3f11e71 Fix various compiler warnings 2019-03-15 10:34:57 -07:00
Robert Escriva
268093a96d Adjust all includes to be relative to the root.
Remove the use of relative paths.  A header at foo/bar.h could be included by
files under foo/ with "bar.h", but would be included everywhere else as
"foo/bar.h".  Adjust so that every include references such a header with the
latter form.

Signed-off-by: Robert Escriva <rescriva@dropbox.com>
2018-10-19 17:35:33 +00:00
A.J. Beamon
fe956bc35a Address review comments 2018-06-26 14:37:21 -07:00
A.J. Beamon
5848055620 More line ending fixes 2018-05-31 13:44:38 -07:00
A.J. Beamon
f1dea3bed9 Rework the formatter a bit and extract out the writing logic so that it could be pluggable. 2018-05-31 13:27:35 -07:00