54 Commits

Author SHA1 Message Date
Yi Wu
994b8c92f8
Add option to limit resident memory and remove default memory limit (#6719)
Changing `memory` option to limit resident memory instead of virtual memory, in config file and fdbserver/fdbbackup/fdbcli command-line argument. Since `rlimit` doesn't support limiting virtual memory, the current implementation have both of fdbmonitor and the fdbserver/fdbbackup process checking process RSS periodically and kill and restart the process if the limit is exceeded.

Adding a new `memory_vsize` option to limit virtual memory, if backward-compatible behavior is desired.

closes #6671, closes #6672
2022-04-06 20:06:24 -07:00
sfc-gh-tclinkenbeard
a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
sfc-gh-tclinkenbeard
5b8f4d9be1 Move SimpleIni.h to fdbclient 2022-02-07 13:31:04 -08:00
A.J. Beamon
f29f487823
Unify flags (#25)
* Unify flags implementation and change help text in backup.actor.cpp
Description

Testing

* Keep LOG_GROUP unchanged

Description

Testing

* Transfer the hyphens to underscores for internal options and user's input, EXCEPT leading hyphens

Description

Testing

* Use a deep copy of the user's input flag to do the match

Description

Testing

* Convert the _ to - in Option arrays of backup.actor.cpp

Description

Testing

* Transter _ to - for files:
        TLSConfig.actor.h, fdbcli.actor.cpp, fdbserver.actor.cpp, FileConverter.h, FileConverter.cpp

Description

Testing

* Change another way to unify flag: using SO_O_ICASE_HYPHEN_AND_UNDERSCORE to determine whether we do the conversion in function IsEqual

Description

Testing

* Change the config command's name from SO_O_ICASE_HYPHEN_AND_UNDERSCORE to SO_O_HYPHEN_TO_UNDERSCORE

Description

Testing

* Update the comment for the SO_O_HYPHEN_TO_UNDERSCORE

Description

Testing

* Fix left underscore in SOption arrays

Description

Testing

* Convert _ to - in several files for commands

Description

Testing

* Make the FDBService and fdbmonitor backward compatible

Description

Testing

* Fix bugs about pointers

Description

Testing

* Check underscore and hyphen at the same time for --knob_, --localily_ and --test_
And fix bugs in fdbmonitor and FDBService
Description

Testing

* Simplify the function in fdbmonitor and FDBService about retrieving arguments.
And fix some documents in masterserver.actor.cpp

Description

Testing

* Convert _ to - for knob in the setKnob functions

Description

Testing

* Convert - to _ in the setKnob functions

Description
Since key in the knob related maps only contain _

Testing

* Rename varialbe name in the fdbmonitor and FDBService for clarification

Description

Testing

Co-authored-by: Chang Liu <chang.liu@snowflake.com>
2021-12-14 08:44:39 -08:00
sfc-gh-tclinkenbeard
90ced244eb Fix -Wunused-but-set-variable warnings 2021-12-01 18:15:53 -08:00
A.J. Beamon
1a5efa004a Include the errno details if execv fails in fdbmonitor 2021-09-28 10:02:45 -07:00
sfc-gh-tclinkenbeard
b9a22a61ef Fix many -Wreorder-ctor warnings 2021-07-23 17:33:18 -07:00
A.J. Beamon
5832e61c0c Make fdbmonitor IDs unique per section rather than globally unique 2021-07-13 16:13:48 -04:00
A.J. Beamon
c49a095677 Remove some misleading wording 2021-07-13 09:15:54 -04:00
Daniel Smith
5529c96e51 Clean up a few memory leaks 2021-06-25 10:38:54 -04:00
sfc-gh-tclinkenbeard
f28ac955c3 Remove unnecessary temporary objects while growing objects of type std::vector<std::pair<A, B>> 2021-05-10 16:32:50 -07:00
FDB Formatster
df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
sfc-gh-tclinkenbeard
9a7b6eb51c Replace NULL with nullptr in fdbmonitor 2020-09-20 11:33:17 -07:00
A.J. Beamon
d128252e90 Merge release-6.3 into master 2020-05-22 09:25:32 -07:00
Alex Miller
78d644f673
Merge branch 'master' into bugfixes/cmake-includes 2020-05-04 02:11:46 -07:00
Dave Cottlehuber
28e58c672a fdbmonitor: ifdef config path location 2020-04-30 20:11:43 +00:00
Dave Cottlehuber
b2eb93f5f4 fdbmonitor: define O_EVTONLY for FreeBSD 2020-04-30 20:11:43 +00:00
Dave Cottlehuber
5064cc776b fdbmonitor: update headers & libraries 2020-04-30 20:11:43 +00:00
Markus Pilman
e4611e8ae4 fix versions.h stupidity 2020-04-06 10:28:55 -07:00
Markus Pilman
8b5780c36c don't include source and binary dir
This forces users to use include paths from the sources root.

So `#include "Arena.h"` won't work anymore, only
`#include "flow/Arena.h"` will.
2020-04-06 10:13:49 -07:00
Austin Seipp
0b263fd856 fdbmonitor: fix some print/scan format warnings
Signed-off-by: Austin Seipp <aseipp@pobox.com>
2019-05-06 13:35:29 -07:00
Andrew Noyes
6207d724f8 Fix all -Wunused-variable warnings 2019-04-15 18:13:00 -07:00
Stephen Atherton
382a7bdc5f Changed behavior regarding ~ and ~user paths to treat them as unresolvable symbolic links. 2019-03-22 16:21:12 -07:00
Stephen Atherton
524a666290 Added back previous handling of ~ in paths, with the improvement that it only treats ~ as special if it is the first character. 2019-03-22 11:44:46 -07:00
Stephen Atherton
6c42e42334 Fix compiler complaint about ignored return codes being ignored. Added commented call to testPathOps() for convenience. 2019-03-21 18:05:30 -07:00
Stephen Atherton
a6b598f236 Missing include. 2019-03-21 17:51:00 -07:00
Stephen Atherton
c6d96498ca parentDirectory() is now based on popPath(). Bug fix, abspath() would prepend current working directory even when not resolving symlinks. Added more unit tests. Ported path operation unit tests to fdbmonitor() since the path manipulation function implementations are significantly different. Clarified some comments. The flow project version of abspath() does not allow resolveLinks to be false, for now, because behavior of this on Windows is not well thought out or tested. 2019-03-21 16:56:36 -07:00
Stephen Atherton
644a88e8b4 Updated fdbmonitor to use modified/minimal implementations of new abspath() and parentDirectory() implementations from flow. 2019-03-20 23:18:30 -07:00
Stephen Atherton
2efb6f4c0d Added cleanPath() which puts a path in a canonical form without .., ., or duplicate separators without using the filesystem or resolving symbolic links. absPath() redefined to use cleanPath() so it will return the same result for a path without symbolic links regardless of whether or not the path actually exists. Redefined parentDirectory() to use absPath() and error on certain inputs. Added comments describing behavior of these functions, and added a unit test which verbosely tests many inputs to them. 2019-03-15 23:54:33 -07:00
A.J. Beamon
809861ee88 Merge branch 'master' into fix-parent-directory
# Conflicts:
#	flow/Platform.cpp
2019-03-15 10:51:12 -07:00
mpilman
6da5971e79 Guard all versions.h to not break old WIN32 build 2019-02-08 16:06:00 -08:00
mpilman
7a858b902d Most of FDB compiling on Windows with cmake 2019-02-07 15:37:04 -08:00
Bhaskar Muppana
448e654a03 Fix Make build. 2019-01-03 13:14:02 -08:00
mpilman
a31df1b0a6 Addressed comments from pull requests
- Moved some larger sections in CMakeLists.txt into separate files
- Fixed an include issue on OS X
- Fixed boost version
- Use PROJECT_VERSION by default instead of using versions.target
2019-01-02 13:32:26 -08:00
Evan Tschannen
4e54690005 Merge branch 'release-6.0'
# Conflicts:
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/MoveKeys.actor.cpp
2018-11-12 20:26:58 -08:00
Stephen Atherton
3bd66217c6 Bug fix, kill_process() modifies the id <-> pid maps so to kill all ids one by one the loop iterator must be advanced before the kill. 2018-11-08 16:21:50 -08:00
Robert Escriva
268093a96d Adjust all includes to be relative to the root.
Remove the use of relative paths.  A header at foo/bar.h could be included by
files under foo/ with "bar.h", but would be included everywhere else as
"foo/bar.h".  Adjust so that every include references such a header with the
latter form.

Signed-off-by: Robert Escriva <rescriva@dropbox.com>
2018-10-19 17:35:33 +00:00
Stephen Atherton
4488ac922c Typo which was missed because of a local make issue causing success without recompile. 2018-10-08 10:51:10 -07:00
Stephen Atherton
409a9b66b6 FDBMonitor will only kill its entire process group on exit if it started its own process group with setsid() (as with daemonize mode), otherwise it will kill its processes individually. 2018-10-08 10:42:11 -07:00
Stephen Atherton
e17c04f666 Changed assignments to substr() to resize() for clarity/simplicity. 2018-07-02 14:25:24 -07:00
A.J. Beamon
7378ec85f0 Fixes parentDirectory to work when our path is a child of the root directory. Also works when the path has trailing slashes. 2018-06-26 16:30:47 -07:00
Alec Grieser
752deb07a1
fix fdbmonitor help message output ; fix spelling error Ratekeeper.actor.cpp 2018-05-07 16:19:50 -07:00
Alec Grieser
0bae9880f1 remove trailing whitespace from our copyright headers ; fixed formatting of python setup.py 2018-02-21 10:25:11 -08:00
A.J. Beamon
8891804b1b Add some comments to watch setting in fdbmonitor. 2018-01-04 11:31:01 -08:00
A.J. Beamon
c0d37864cf Remove the watch on parent of missing directory if we detect that the directory has just shown up. 2018-01-04 11:30:41 -08:00
A.J. Beamon
d52280b628 Merge branch 'release-5.1' into fdbmonitor-deconfigure-on-delete
# Conflicts:
#	fdbmonitor/fdbmonitor.cpp
2018-01-04 11:23:35 -08:00
A.J. Beamon
6f452ae9fa Remove unused variable 2018-01-04 10:30:02 -08:00
A.J. Beamon
9d83501800 fdbmonitor now deconfigures processes when the config file is deleted. Depending on the specified settings, running processes may or may not be killed in that case. If they are not killed, then they are rejoined with the config if it reappears.
In this implementation, the deletion or moving of a parent directory does not trigger an immediate reload of the configuration, but the change will be detected the next time the configuration is reloaded for other reasons.
2017-11-30 13:57:05 -08:00
A.J. Beamon
5e984a366a Updates to fdbmonitor logging so that each line contains a timestamp, severity, logGroup, and process (e.g. fdbmonitor or fdbserver.4689, etc.). Remove usages of perror. 2017-10-31 11:07:06 -07:00
A.J. Beamon
c1bc355306 Add support for retrying a process if fork fails. The HUP signal now causes configuration to be reloaded and timeouts to be reset. A little refactoring to make this easier. 2017-10-05 08:23:52 -07:00