56 Commits

Author SHA1 Message Date
Andrew Noyes
da1ffebcb0
Improve test harness logging when there are no trace files (#7785)
* Log OldBinary even if there are no trace files

DeterminismCheck and OldBinary attributes don't actually depend on
information in the ProgramStart event, so we can add them
unconditionally.

* Add JoshuaSeed attribute to Test element in test harness

* Add NoTraceFilesFound event in test harness

There's already something similar: NoTraceFileGenerated. It appears that
the original author only wants to log that if the process exited 0
though. I'm not sure what the reason for that is so I think it's safer
to add a new event. This will make it more clear if say an old binary is
corrupt.
2022-08-03 17:14:33 -07:00
Markus Pilman
92b1d1f647 Don't fail TestHarness if comment doesn't exist 2022-07-28 11:03:09 -06:00
Markus Pilman
3327ab395b Change test harness to include comments 2022-07-19 15:41:53 -06:00
Andrew Noyes
17140a2645 Always run valgrind on the binary under test (if valgrind is enabled) 2022-05-10 14:46:22 -07:00
Andrew Noyes
68c03a7e32
Jemalloc integration fixes (#6626)
* Set default for USE_JEMALLOC initially in ConfigureCompiler

Instead of trying to change the value later on. This fixes the valgrind
build, which was previously incorrectly getting jemalloc involved.

* Check aligned_alloc result for null

And OOM if so - don't assert

* Check that we can allocate magazines with no internal fragmentation

We may want to do this so that the jemalloc heap profiler has some
knowledge of FastAlloc

* Populate TestFile field for noSim tests in TestHarness

* Remove handling for nonexistent "ActualRun"
2022-03-17 15:17:27 -07:00
A.J. Beamon
00277140ec Non-simulated test harness runs don't check the unseed. 2022-03-17 11:09:14 -07:00
Andrew Noyes
00ceaeb1b3 Move generated part of Program.cs to its own file
This way editors/tools will understand that it's a c sharp file
2022-02-07 13:28:06 -08:00
A.J. Beamon
30e2c2d9a6 Don' use new-style arguments in test harness. 2021-12-14 12:31:12 -08:00
A.J. Beamon
f24adc7b6a Fix a bunch of places where we used old-style arguments. Allow hyphens for profiler args. 2021-12-14 09:59:14 -08:00
Aaron Molitor
77db63274b use FDB_VERSION in lieu of PROJECT_VERSION or CMAKE_PROJECT_VERSION
write FDB_VERSION to a file to avoid regex golf in other build scripts
2021-11-29 15:11:20 -08:00
Chaoguang Lin
0d1382eeba Add comments for the change 2021-09-03 14:32:27 -07:00
Chaoguang Lin
56ed0f3b9c Return if no available binary in the given range 2021-09-03 14:21:16 -07:00
Chaoguang Lin
8622363395 Simplify the condition and update the comment 2021-09-02 18:51:39 -07:00
Chaoguang Lin
3ad620b2df Update comment to be more clear 2021-09-02 18:13:04 -07:00
Chaoguang Lin
e7028b7915 Handle the edge case correctly when the upper bound version is less than the binary verion 2021-09-02 17:42:46 -07:00
Chaoguang Lin
82e1e850c2 Add syntax until_*.*.* in restarting test to specify upper bound of binary version 2021-09-02 14:07:15 -07:00
Jingyu Zhou
fffb0d6fb9
Merge pull request #5305 from sajjadrahnama/test_harness_buggify_fault_injection
Test harness Buggify/FaultInjection argument
2021-07-30 09:15:59 -07:00
Sajjad
5b191fccfe
Update contrib/TestHarness/Program.cs.cmake - Minor change
Co-authored-by: Jingyu Zhou <jingyuzhou@gmail.com>
2021-07-29 23:18:15 -07:00
Sajjad Rahnama
12e1a5fe92 TestHarness Buggify/FaultInjection Enable/Disable - Update Old Binaries arguments 2021-07-29 11:26:14 -07:00
Steve Atherton
b3e22ad573 Move stderr exception checking so that ignored output does not count against the error limit. 2021-07-26 18:10:03 -07:00
Sajjad Rahnama
c7ef116c12 TestHarness Buggify/FaultInjection Enable/Disable 2021-07-26 16:44:10 -07:00
Steve Atherton
ff8a1e0ed2 Ignore warning from valgrind about F_SET_RW_HINT usage. 2021-07-26 16:37:55 -07:00
Oleg Samarin
97bcb16000 Fixed formatting 2021-07-14 19:24:04 +03:00
Oleg Samarin
e97fb31d9b Added some comment 2021-07-14 17:46:15 +03:00
Oleg Samarin
04549c59ed Fixed 'NoTraceFileGenerated' error in simulation tests when fdbserver process could not start 2021-07-14 17:41:08 +03:00
Daniel Smith
efdda3cd0e Revert "Revert "Add support for running non-simulation tests on Joshua"" 2021-06-15 14:15:02 -04:00
Daniel Smith
549cf0512b
Revert "Add support for running non-simulation tests on Joshua" 2021-06-14 19:23:56 -04:00
Steve Atherton
3a70012c08
Merge pull request #4971 from Daniel-B-Smith/test-nonsim
Add support for running non-simulation tests on Joshua
2021-06-14 13:56:24 -07:00
Daniel Smith
8af0affeff Add support for running non-simulation tests on Joshua 2021-06-14 15:00:45 -04:00
Andrew Noyes
948f4993b9 Convert to 32 bit int 2021-06-09 17:07:45 -07:00
Andrew Noyes
04694a6f70 Respect JOSHUA_SEED if set
Closes #4913
2021-06-09 16:21:50 -07:00
Josh Slocum
d067ca507b Surfacing non-fatal parse errors in Test Harness output 2021-05-19 20:04:12 +00:00
Andrew Noyes
540c6bb709 Ignore expected ASAN message 2021-02-18 19:18:07 +00:00
A.J. Beamon
0650ff1a41 Increase the maximum length of the error strings logged by test harness. Make it more obvious when errors are being truncated. 2021-02-05 15:54:51 -08:00
Andrew Noyes
58b4a9b907 Resolve conflicts 2021-01-21 20:59:44 +00:00
Andrew Noyes
281151ff20 Merge remote-tracking branch 'upstream/release-6.2' into anoyes/merge-6.2-to-6.3
Commit conflict markers for now
2021-01-21 17:54:26 +00:00
Andrew Noyes
ddb5b0b050 Backport correctness package 2021-01-15 00:34:47 +00:00
Alex Miller
16108c0795 Handle versions being >1 digits. 2020-10-27 13:12:42 -07:00
Alex Miller
ee45ed1c11 TestHarness should check --version before choosing an old binary.
This allows joshua to run on binaries older than the most recent version,
because restarting/upgrade tests won't start with a newer version than what is
under test.
2020-10-26 19:19:45 -07:00
Alvin Moore
66a41089dc
Increment TestHarness version
Co-authored-by: Markus Pilman <markus.pilman@snowflake.com>
2020-08-07 14:28:27 -04:00
Alvin Moore
c82a81b124 Fixed whitespace 2020-08-07 13:50:49 -04:00
Alvin Moore
6ec05e1fd8 Removed unneeded include for TestHarness 2020-08-07 13:50:01 -04:00
Alvin Moore
12b51f7889 Added separate function to display version information for TestHarness 2020-08-07 12:44:21 -04:00
Alvin Moore
bc0ef5d0d7 Fixed the name of the TestHarness cmake file 2020-08-07 12:40:52 -04:00
Alvin Moore
d5b39f649c Updated cmake to work with TestHarness source file 2020-08-07 12:37:27 -04:00
Alvin Moore
c07aaf2d72 Removed Application config
Made TestHarness source file a cmake file
2020-08-07 12:03:35 -04:00
Alvin Moore
0f6a01aef4 Added TestHarness application config 2020-08-07 10:57:14 -04:00
Alvin Moore
005b59cc84 Added support for storing and displaying version within TestHarness 2020-08-07 10:46:38 -04:00
sfc-gh-tclinkenbeard
369d1be562 Fixed test harness bug 2020-07-12 16:06:23 -07:00
sfc-gh-tclinkenbeard
949a17359a Added to_6.3.0 downgrade test to test harness 2020-07-09 14:29:43 -07:00